From patchwork Wed Oct 31 18:39:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Boccassi X-Patchwork-Id: 47632 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6F1FE2BF9; Wed, 31 Oct 2018 19:40:08 +0100 (CET) Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by dpdk.org (Postfix) with ESMTP id 9C6C2FEB; Wed, 31 Oct 2018 19:40:06 +0100 (CET) Received: by mail-wr1-f68.google.com with SMTP id u1-v6so17662982wrn.0; Wed, 31 Oct 2018 11:40:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sV/fdltn3K+Syt8GMuEP7rhL35P5Ajz4wka9sQSSruA=; b=mEFi/qACTVabM5bcWujkH6Ezoz3Z8EloWUgpxe4rLOtRW7A/LGE1gIqs8auIFLDaOD Gi6/hfF2qCXuTE2VmoIYHDpvq9OhFStuT+lDsN2a5lbApiP1e/Tp0/jpnX1FUWKkWJhq sQheCs45Dkqskx1hNdiBpFbHQhSnKmN7lOaFJE+i3rdA4Th95jMBci0lMgh2vuABtl/7 rFxKeJ1pDtYYtL2v+3su2fGFjJsACQ5QLihyVri+ncoBb1AYeQnkuf14XbgpJEVIcJTZ 83TfYW9qZRmYr1G3RQ9v69QyuTBDO8rBFTjtr8EnNdkyfuZcAhoUR1MT9/ENbFKVzKlf /iBQ== X-Gm-Message-State: AGRZ1gIzsOrsXLTZVsHMWpbDVNJrLGdE7K64G6BqZg/wAoObWA2v+M1M +J5Yu3g8pHqLDUaP35Hqa5qr3tOU X-Google-Smtp-Source: AJdET5c777bnwDhNBW26V7Uov/9OKCjn8L/nfmVhSIJilwNraHXfBa8x0UmdUIUmoFj6QIBj5e+2dw== X-Received: by 2002:adf:edcf:: with SMTP id v15-v6mr3684237wro.182.1541011205637; Wed, 31 Oct 2018 11:40:05 -0700 (PDT) Received: from localhost ([2a01:4b00:f419:6f00:8361:8946:ba2b:d556]) by smtp.gmail.com with ESMTPSA id m141-v6sm131573wmd.14.2018.10.31.11.40.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 31 Oct 2018 11:40:04 -0700 (PDT) From: Luca Boccassi To: dev@dpdk.org Cc: yongwang@vmware.com, 3chas3@gmail.com, bruce.richardson@intel.com, anatoly.burakov@intel.com, thomas@monjalon.net, llouis@vmware.com, Luca Boccassi , stable@dpdk.org, Brian Russell Date: Wed, 31 Oct 2018 18:39:43 +0000 Message-Id: <20181031183945.29509-1-bluca@debian.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20180816135032.28283-1-bluca@debian.org> References: <20180816135032.28283-1-bluca@debian.org> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v3 1/3] net/virtio: register/unregister intr handler on start/stop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Register and unregister the virtio interrupt handler when the device is started and stopped. This allows a virtio device to be hotplugged or unplugged. Fixes: c1f86306a026 ("virtio: add new driver") Cc: stable@dpdk.org Signed-off-by: Brian Russell Signed-off-by: Luca Boccassi Reviewed-by: Maxime Coquelin --- drivers/net/virtio/virtio_ethdev.c | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index 10a7e3fcc..da8717726 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -1679,11 +1679,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev) if (ret < 0) goto out; - /* Setup interrupt callback */ - if (eth_dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) - rte_intr_callback_register(eth_dev->intr_handle, - virtio_interrupt_handler, eth_dev); - return 0; out: @@ -1706,11 +1701,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev) eth_dev->tx_pkt_burst = NULL; eth_dev->rx_pkt_burst = NULL; - /* reset interrupt callback */ - if (eth_dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) - rte_intr_callback_unregister(eth_dev->intr_handle, - virtio_interrupt_handler, - eth_dev); if (eth_dev->device) rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev)); @@ -1969,6 +1959,12 @@ virtio_dev_start(struct rte_eth_dev *dev) dev->data->dev_conf.intr_conf.rxq) { virtio_intr_disable(dev); + /* Setup interrupt callback */ + if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) + rte_intr_callback_register(dev->intr_handle, + virtio_interrupt_handler, + dev); + if (virtio_intr_enable(dev) < 0) { PMD_DRV_LOG(ERR, "interrupt enable failed"); return -EIO; @@ -2078,9 +2074,17 @@ virtio_dev_stop(struct rte_eth_dev *dev) PMD_INIT_LOG(DEBUG, "stop"); rte_spinlock_lock(&hw->state_lock); - if (intr_conf->lsc || intr_conf->rxq) + if (intr_conf->lsc || intr_conf->rxq) { virtio_intr_disable(dev); + /* Reset interrupt callback */ + if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) { + rte_intr_callback_unregister(dev->intr_handle, + virtio_interrupt_handler, + dev); + } + } + hw->started = 0; memset(&link, 0, sizeof(link)); rte_eth_linkstatus_set(dev, &link); From patchwork Wed Oct 31 18:39:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Boccassi X-Patchwork-Id: 47633 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E857D4C8A; Wed, 31 Oct 2018 19:40:12 +0100 (CET) Received: from mail-wm1-f65.google.com (mail-wm1-f65.google.com [209.85.128.65]) by dpdk.org (Postfix) with ESMTP id EA48F34EF; Wed, 31 Oct 2018 19:40:08 +0100 (CET) Received: by mail-wm1-f65.google.com with SMTP id y144-v6so15423453wmd.4; Wed, 31 Oct 2018 11:40:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s9whBD7MJWJWV5zd4ST0bIXPe+U9+snVxwLA9B+UJo4=; b=M9aHn5T7LhkB1jllvCZflo1IuUuHUvB1JX/HeIOUD1BOxKYprIglB0v4fuXScWuakU papAmR62L6yLEMDW4VzDCp4sjhTADxgGad41vCtp26yueciNuX18pY58OkVaE5j2tOrO j2V/4KrbzW6LM+2d4Pg9NYC5UPSlhjqERcZMLyXlfQYWz9ZTUOp1bq4KOgczDC6UKv5T LK4bBqbA8bIB5x11R5nMbLyoUojC4UWtUmTmxAdrS8nPHsHCHgcqBqtGluB3Ht9CPU55 1Q37tfFBsjBrYwYhxLXlGcEXYlk6ffdUy2xefyEfAlDdOZkixiZWiVM3Gyy2u0+fIyb7 HHBg== X-Gm-Message-State: AGRZ1gIrl4xVb9DcVhgIExtlR78GlwqeNtIWkbjKsQvVVJ7KaMO2P56S JGt3SAA+Aq6nRDJxr7DXq3HZ3J+A X-Google-Smtp-Source: AJdET5edwoyuQBPXCie1EFQhzoGNizcm82PpVfcaFjRh/p/EOX+ySYuGxCYRNTjbgifMKebHtLtEbw== X-Received: by 2002:a1c:8c8c:: with SMTP id o134-v6mr3253411wmd.35.1541011208051; Wed, 31 Oct 2018 11:40:08 -0700 (PDT) Received: from localhost ([2a01:4b00:f419:6f00:8361:8946:ba2b:d556]) by smtp.gmail.com with ESMTPSA id b71-v6sm22794753wma.13.2018.10.31.11.40.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 31 Oct 2018 11:40:07 -0700 (PDT) From: Luca Boccassi To: dev@dpdk.org Cc: yongwang@vmware.com, 3chas3@gmail.com, bruce.richardson@intel.com, anatoly.burakov@intel.com, thomas@monjalon.net, llouis@vmware.com, Luca Boccassi , stable@dpdk.org, Brian Russell Date: Wed, 31 Oct 2018 18:39:44 +0000 Message-Id: <20181031183945.29509-2-bluca@debian.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181031183945.29509-1-bluca@debian.org> References: <20180816135032.28283-1-bluca@debian.org> <20181031183945.29509-1-bluca@debian.org> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v3 2/3] net/vmxnet3: fix vmxnet3 dev_uninit() hot-unplug X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The vmxnet3 driver can't call back into dev_close(), and possibly dev_stop(), in dev_uninit(). When dev_uninit() is called, anything that those routines would want to clean up has already been released. Further, for complete cleanup, it is necessary to release any of the queue resources during dev_close(). This allows a vmxnet3 device to be hot-unplugged without leaking queues. Also set RTE_ETH_DEV_CLOSE_REMOVE on close so that the port resources can be deallocated. Return EBUSY if remove is called before stop. Fixes: dfaff37fc46d ("vmxnet3: import new vmxnet3 poll mode driver implementation") Cc: stable@dpdk.org Signed-off-by: Brian Russell Signed-off-by: Luca Boccassi --- v2: add back extra close() call in uninit() for buggy applications as requested by the reviewers, and add debug log noting the issue v3: remove extra close() call in uninit() as requested, and return -EBUSY instead. set RTE_ETH_DEV_CLOSE_REMOVE in close(). drivers/net/vmxnet3/vmxnet3_ethdev.c | 43 +++++++++++++++++++++++----- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c index 41bcd450a..84acd9dbb 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c @@ -360,8 +360,10 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev) if (rte_eal_process_type() != RTE_PROC_PRIMARY) return 0; - if (hw->adapter_stopped == 0) - vmxnet3_dev_close(eth_dev); + if (hw->adapter_stopped == 0) { + PMD_INIT_LOG(DEBUG, "Device has not been closed."); + return -EBUSY; + } eth_dev->dev_ops = NULL; eth_dev->rx_pkt_burst = NULL; @@ -805,7 +807,7 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) PMD_INIT_FUNC_TRACE(); if (hw->adapter_stopped == 1) { - PMD_INIT_LOG(DEBUG, "Device already closed."); + PMD_INIT_LOG(DEBUG, "Device already stopped."); return; } @@ -829,7 +831,6 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) /* reset the device */ VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_RESET_DEV); PMD_INIT_LOG(DEBUG, "Device reset."); - hw->adapter_stopped = 0; vmxnet3_dev_clear_queues(dev); @@ -839,6 +840,30 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) link.link_speed = ETH_SPEED_NUM_10G; link.link_autoneg = ETH_LINK_FIXED; rte_eth_linkstatus_set(dev, &link); + + hw->adapter_stopped = 1; +} + +static void +vmxnet3_free_queues(struct rte_eth_dev *dev) +{ + int i; + + PMD_INIT_FUNC_TRACE(); + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + void *rxq = dev->data->rx_queues[i]; + + vmxnet3_dev_rx_queue_release(rxq); + } + dev->data->nb_rx_queues = 0; + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + void *txq = dev->data->tx_queues[i]; + + vmxnet3_dev_tx_queue_release(txq); + } + dev->data->nb_tx_queues = 0; } /* @@ -847,12 +872,16 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) static void vmxnet3_dev_close(struct rte_eth_dev *dev) { - struct vmxnet3_hw *hw = dev->data->dev_private; - PMD_INIT_FUNC_TRACE(); vmxnet3_dev_stop(dev); - hw->adapter_stopped = 1; + vmxnet3_free_queues(dev); + + /* + * flag to rte_eth_dev_close() that it should release the port resources + * (calling rte_eth_dev_release_port()) in addition to closing it. + */ + dev->data->dev_flags |= RTE_ETH_DEV_CLOSE_REMOVE; } static void From patchwork Wed Oct 31 18:39:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Boccassi X-Patchwork-Id: 47634 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id E8D684F90; Wed, 31 Oct 2018 19:40:15 +0100 (CET) Received: from mail-wr1-f67.google.com (mail-wr1-f67.google.com [209.85.221.67]) by dpdk.org (Postfix) with ESMTP id D73384C8A; Wed, 31 Oct 2018 19:40:10 +0100 (CET) Received: by mail-wr1-f67.google.com with SMTP id w5-v6so17641428wrt.2; Wed, 31 Oct 2018 11:40:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xPQbXM4Yi9ZNe0BBhg9oIKd2QtSOOYNqjilAj1iluTw=; b=mxE4+prQW2JYpCmpG060cjz3aQjvLiMRO/D6x8o2SkWgFtr414kRNsoSL9slIhs6SV 6OAgRsryBwMi70CKW/XWm96wA29tjmdze/fUnwWinkZSBdB0exsW+U8k/D/itk2Phqr2 4ImARjgKrca4PobM+493o6UAFKzqGk367tqLvhAwy/GwFk8FHpFnxSsfnzOlubVURqr4 A2t0c9GrvB36ydPbRRxJ+RrYBFYW6AbOhQtYQ5VL6p7mYwEXAcjslUawbNApoj4o8bgp 1mm9AAs/Uy8fprEhQJi2NUrBzwCKVcsrZvWPsxQRk/lealhmXANdPOozXLkktZy23R+W 0+Fw== X-Gm-Message-State: AGRZ1gIr1ZEtiz9OxnEU3PRmB2WVJ3ANAAnvel/9q9psd2dPbt6bp/9g ToMI8hEf5402fY5qCYN8svmq/GnA X-Google-Smtp-Source: AJdET5dM5HpvBuxAbVtRrtvu7n6xnBkRoFB3+CqKlwxS9gI7BJURpwFZ2a9lJdMvkmCU9On6IX9gDA== X-Received: by 2002:adf:90af:: with SMTP id i44-v6mr3617606wri.77.1541011210115; Wed, 31 Oct 2018 11:40:10 -0700 (PDT) Received: from localhost ([2a01:4b00:f419:6f00:8361:8946:ba2b:d556]) by smtp.gmail.com with ESMTPSA id l140-v6sm38538116wmb.24.2018.10.31.11.40.08 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 31 Oct 2018 11:40:09 -0700 (PDT) From: Luca Boccassi To: dev@dpdk.org Cc: yongwang@vmware.com, 3chas3@gmail.com, bruce.richardson@intel.com, anatoly.burakov@intel.com, thomas@monjalon.net, llouis@vmware.com, Luca Boccassi , stable@dpdk.org, Brian Russell Date: Wed, 31 Oct 2018 18:39:45 +0000 Message-Id: <20181031183945.29509-3-bluca@debian.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181031183945.29509-1-bluca@debian.org> References: <20180816135032.28283-1-bluca@debian.org> <20181031183945.29509-1-bluca@debian.org> MIME-Version: 1.0 Subject: [dpdk-dev] [PATCH v3 3/3] eal/linux: handle uio read failure in interrupt handler X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" If a device is unplugged while an interrupt is pending, the read call to the uio device to remove it from the poll wait list can fail resulting in it being continually polled forever. This change checks for the read failing and if so, unregisters the device as an interrupt source and causes the wait list to be rebuilt. This race has been reported and observed in production. Fixes: 0a45657a6794 ("pci: rework interrupt handling") Cc: stable@dpdk.org Signed-off-by: Brian Russell Signed-off-by: Luca Boccassi --- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_interrupts.c b/lib/librte_eal/linuxapp/eal/eal_interrupts.c index 39252a887..cbac451e1 100644 --- a/lib/librte_eal/linuxapp/eal/eal_interrupts.c +++ b/lib/librte_eal/linuxapp/eal/eal_interrupts.c @@ -700,7 +700,7 @@ eal_intr_process_interrupts(struct epoll_event *events, int nfds) bool call = false; int n, bytes_read; struct rte_intr_source *src; - struct rte_intr_callback *cb; + struct rte_intr_callback *cb, *next; union rte_intr_read_buffer buf; struct rte_intr_callback active_cb; @@ -780,6 +780,23 @@ eal_intr_process_interrupts(struct epoll_event *events, int nfds) "descriptor %d: %s\n", events[n].data.fd, strerror(errno)); + /* + * The device is unplugged or buggy, remove + * it as an interrupt source and return to + * force the wait list to be rebuilt. + */ + rte_spinlock_lock(&intr_lock); + TAILQ_REMOVE(&intr_sources, src, next); + rte_spinlock_unlock(&intr_lock); + + for (cb = TAILQ_FIRST(&src->callbacks); cb; + cb = next) { + next = TAILQ_NEXT(cb, next); + TAILQ_REMOVE(&src->callbacks, cb, next); + free(cb); + } + free(src); + return -1; } else if (bytes_read == 0) RTE_LOG(ERR, EAL, "Read nothing from file " "descriptor %d\n", events[n].data.fd);