From patchwork Wed Sep 19 12:57:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Boccassi X-Patchwork-Id: 44909 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id EFA6C1B53; Wed, 19 Sep 2018 14:58:13 +0200 (CEST) Received: from mail-wr1-f68.google.com (mail-wr1-f68.google.com [209.85.221.68]) by dpdk.org (Postfix) with ESMTP id 85356A49 for ; Wed, 19 Sep 2018 14:58:13 +0200 (CEST) Received: by mail-wr1-f68.google.com with SMTP id y8-v6so2049435wrh.7 for ; Wed, 19 Sep 2018 05:58:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Bp86lqXLR1c8Vsh7ycbEg1XVFB7NiysjDh0rE9VoSP8=; b=YnvdHdFsBP/ThFMfUXwfvZkzfIxjQlRaRZdfGyQYoKQS6PDlNs0K0BXBezw4amwdv2 +1Z9lDvY0J1o8ahUfmePhcFcP9rugTF3brZykcOPyDC2zL+BRHJumjKBjtrMSBvs/BtW tpk7ef6TQyv6ctA0kQUOpBPFXtWDM6rDhugOgFs9dOdntnROaiou/7nIcWpgOj7B5ePa 4o8oOTTnP/gZhuVsBQBrefcVWa7qZqHBpqvnrvJw8opjT3kvAwa/xIbeKIfl1Z/rENj4 ylWhVwTq9ipjtW06PMJDeevzH/+hZNvxh48aNt6CL+P751+RVJtQmn0lMMrDZ6ekeXaw iMcA== X-Gm-Message-State: APzg51AJeOXfktO1efGcDqNCiAKdiRImeWJqPmNRn0BSb3H2GdxuUVb2 OLJlNTNT1S8qhEX+ZD/hGHuZ+yAHl5o= X-Google-Smtp-Source: ANB0VdYhTQVsRbpKs65i/Z6hewwCErH0eEUkrIhOCJ8ynvoUbg+In24PuF+O2sIiYoyPrJR9M79CEQ== X-Received: by 2002:adf:facf:: with SMTP id a15-v6mr28907858wrs.74.1537361892489; Wed, 19 Sep 2018 05:58:12 -0700 (PDT) Received: from localhost ([2001:1be0:110d:fcfe:489f:80a9:5d59:c6bd]) by smtp.gmail.com with ESMTPSA id 144-v6sm6093311wma.45.2018.09.19.05.58.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 19 Sep 2018 05:58:10 -0700 (PDT) From: Luca Boccassi To: dev@dpdk.org Cc: maxime.coquelin@redhat.com, tiwei.bie@intel.com, yongwang@vmware.com, 3chas3@gmail.com, bruce.richardson@intel.com, jianfeng.tan@intel.com, anatoly.burakov@intel.com, llouis@vmware.com, brussell@vyatta.att-mail.com Date: Wed, 19 Sep 2018 13:57:55 +0100 Message-Id: <20180919125757.17938-1-bluca@debian.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180816135032.28283-1-bluca@debian.org> References: <20180816135032.28283-1-bluca@debian.org> Subject: [dpdk-dev] [PATCH v2 1/3] net/virtio: register/unregister intr handler on start/stop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Register and unregister the virtio interrupt handler when the device is started and stopped. This allows a virtio device to be hotplugged or unplugged. Fixes: c1f86306a026 ("virtio: add new driver") Cc: stable@dpdk.org Signed-off-by: Brian Russell Signed-off-by: Luca Boccassi Reviewed-by: Maxime Coquelin --- drivers/net/virtio/virtio_ethdev.c | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index b81df0a99f..adc6a30a32 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -1679,11 +1679,6 @@ eth_virtio_dev_init(struct rte_eth_dev *eth_dev) if (ret < 0) goto out; - /* Setup interrupt callback */ - if (eth_dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) - rte_intr_callback_register(eth_dev->intr_handle, - virtio_interrupt_handler, eth_dev); - return 0; out: @@ -1709,11 +1704,6 @@ eth_virtio_dev_uninit(struct rte_eth_dev *eth_dev) rte_free(eth_dev->data->mac_addrs); eth_dev->data->mac_addrs = NULL; - /* reset interrupt callback */ - if (eth_dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) - rte_intr_callback_unregister(eth_dev->intr_handle, - virtio_interrupt_handler, - eth_dev); if (eth_dev->device) rte_pci_unmap_device(RTE_ETH_DEV_TO_PCI(eth_dev)); @@ -1972,6 +1962,12 @@ virtio_dev_start(struct rte_eth_dev *dev) dev->data->dev_conf.intr_conf.rxq) { virtio_intr_disable(dev); + /* Setup interrupt callback */ + if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) + rte_intr_callback_register(dev->intr_handle, + virtio_interrupt_handler, + dev); + if (virtio_intr_enable(dev) < 0) { PMD_DRV_LOG(ERR, "interrupt enable failed"); return -EIO; @@ -2081,9 +2077,17 @@ virtio_dev_stop(struct rte_eth_dev *dev) PMD_INIT_LOG(DEBUG, "stop"); rte_spinlock_lock(&hw->state_lock); - if (intr_conf->lsc || intr_conf->rxq) + if (intr_conf->lsc || intr_conf->rxq) { virtio_intr_disable(dev); + /* Reset interrupt callback */ + if (dev->data->dev_flags & RTE_ETH_DEV_INTR_LSC) { + rte_intr_callback_unregister(dev->intr_handle, + virtio_interrupt_handler, + dev); + } + } + hw->started = 0; memset(&link, 0, sizeof(link)); rte_eth_linkstatus_set(dev, &link); From patchwork Wed Sep 19 12:57:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Boccassi X-Patchwork-Id: 44910 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C8E8E3772; Wed, 19 Sep 2018 14:58:16 +0200 (CEST) Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by dpdk.org (Postfix) with ESMTP id 67A9F343C for ; Wed, 19 Sep 2018 14:58:15 +0200 (CEST) Received: by mail-wr1-f65.google.com with SMTP id a108-v6so5679456wrc.13 for ; Wed, 19 Sep 2018 05:58:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oFsaAzc/ia0HQrdN2OIOnfFX+61bMmIKXRVsmdt2e7o=; b=h3rSXcWI4MYu4ybOhmX0E3v5Ee55+4IEqGpx5S6YqTr+tpy5oG869hDACq9lCAoDW5 AXUMXFc2lM2Q3JCSEsaw/ENvuXNQlRwoEBXHUfDQFJeagWyTKUVzfSYlr/DvLhSpDA+n oO97m9VqXVqupKiqOxGxKX5sBq/JLc2qV8aKNWjPrfNTsBkPK4NIkZ0mpZU38f75BBvT ZoSkjtyTbqW2dzXS7ekLQtcjxg+YnqJOjhNtwtxSdT/da+vX9K1cIc3+EkHk6X5HXGQN zX+msUU5iBxja2yMI9iWMppK7N+3ZjMd/o6zy63M0KLuAVcRryn3Ieqdy/aAQf1FqjVP YROQ== X-Gm-Message-State: APzg51Abck2LMtvzM+Y3RqRZO8OnMue319k8EBqbu/LBlxia5e28th6a vxQxIy6dH85aWgrJxEapiR8Q1XSeDew= X-Google-Smtp-Source: ANB0Vdan95qjQgmu7OwbHbp1UyaaibWKkcsrW0rJfv284alH3AbixXM5HeD5L9r+cr/kri2GY2st3Q== X-Received: by 2002:adf:fd12:: with SMTP id e18-v6mr29567140wrr.280.1537361894646; Wed, 19 Sep 2018 05:58:14 -0700 (PDT) Received: from localhost ([2001:1be0:110d:fcfe:489f:80a9:5d59:c6bd]) by smtp.gmail.com with ESMTPSA id o15-v6sm21468205wru.23.2018.09.19.05.58.12 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 19 Sep 2018 05:58:13 -0700 (PDT) From: Luca Boccassi To: dev@dpdk.org Cc: maxime.coquelin@redhat.com, tiwei.bie@intel.com, yongwang@vmware.com, 3chas3@gmail.com, bruce.richardson@intel.com, jianfeng.tan@intel.com, anatoly.burakov@intel.com, llouis@vmware.com, brussell@vyatta.att-mail.com Date: Wed, 19 Sep 2018 13:57:56 +0100 Message-Id: <20180919125757.17938-2-bluca@debian.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180919125757.17938-1-bluca@debian.org> References: <20180816135032.28283-1-bluca@debian.org> <20180919125757.17938-1-bluca@debian.org> Subject: [dpdk-dev] [PATCH v2 2/3] net/vmxnet3: fix vmxnet3 dev_uninit() hot-unplug X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" The vmxnet3 driver can't call back into dev_close(), and possibly dev_stop(), in dev_uninit(). When dev_uninit() is called, anything that those routines would want to clean up has already been released. Further, for complete cleanup, it is necessary to release any of the queue resources during dev_close(). This allows a vmxnet3 device to be hot-unplugged without leaking queues. Fixes: dfaff37fc46d ("vmxnet3: import new vmxnet3 poll mode driver implementation") Cc: stable@dpdk.org Signed-off-by: Brian Russell Signed-off-by: Luca Boccassi --- v2: add back extra close() call in uninit() for buggy applications as requested by the reviewers, and add debug log noting the issue drivers/net/vmxnet3/vmxnet3_ethdev.c | 35 +++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/drivers/net/vmxnet3/vmxnet3_ethdev.c b/drivers/net/vmxnet3/vmxnet3_ethdev.c index f1596ab19d..98e5d01890 100644 --- a/drivers/net/vmxnet3/vmxnet3_ethdev.c +++ b/drivers/net/vmxnet3/vmxnet3_ethdev.c @@ -354,8 +354,10 @@ eth_vmxnet3_dev_uninit(struct rte_eth_dev *eth_dev) if (rte_eal_process_type() != RTE_PROC_PRIMARY) return 0; - if (hw->adapter_stopped == 0) + if (hw->adapter_stopped == 0) { + PMD_INIT_LOG(DEBUG, "Device has not been closed."); vmxnet3_dev_close(eth_dev); + } eth_dev->dev_ops = NULL; eth_dev->rx_pkt_burst = NULL; @@ -802,7 +804,7 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) PMD_INIT_FUNC_TRACE(); if (hw->adapter_stopped == 1) { - PMD_INIT_LOG(DEBUG, "Device already closed."); + PMD_INIT_LOG(DEBUG, "Device already stopped."); return; } @@ -826,7 +828,6 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) /* reset the device */ VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_RESET_DEV); PMD_INIT_LOG(DEBUG, "Device reset."); - hw->adapter_stopped = 0; vmxnet3_dev_clear_queues(dev); @@ -836,6 +837,30 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) link.link_speed = ETH_SPEED_NUM_10G; link.link_autoneg = ETH_LINK_FIXED; rte_eth_linkstatus_set(dev, &link); + + hw->adapter_stopped = 1; +} + +static void +vmxnet3_free_queues(struct rte_eth_dev *dev) +{ + int i; + + PMD_INIT_FUNC_TRACE(); + + for (i = 0; i < dev->data->nb_rx_queues; i++) { + void *rxq = dev->data->rx_queues[i]; + + vmxnet3_dev_rx_queue_release(rxq); + } + dev->data->nb_rx_queues = 0; + + for (i = 0; i < dev->data->nb_tx_queues; i++) { + void *txq = dev->data->tx_queues[i]; + + vmxnet3_dev_tx_queue_release(txq); + } + dev->data->nb_tx_queues = 0; } /* @@ -844,12 +869,10 @@ vmxnet3_dev_stop(struct rte_eth_dev *dev) static void vmxnet3_dev_close(struct rte_eth_dev *dev) { - struct vmxnet3_hw *hw = dev->data->dev_private; - PMD_INIT_FUNC_TRACE(); vmxnet3_dev_stop(dev); - hw->adapter_stopped = 1; + vmxnet3_free_queues(dev); } static void From patchwork Wed Sep 19 12:57:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luca Boccassi X-Patchwork-Id: 44911 X-Patchwork-Delegate: thomas@monjalon.net Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 6031C4C96; Wed, 19 Sep 2018 14:58:18 +0200 (CEST) Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) by dpdk.org (Postfix) with ESMTP id 0F4EA4C90 for ; Wed, 19 Sep 2018 14:58:17 +0200 (CEST) Received: by mail-wr1-f45.google.com with SMTP id y8-v6so2049622wrh.7 for ; Wed, 19 Sep 2018 05:58:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=OoN1luTMKuIv4XtXNZbAo4G+yixREYqmXo693o6dr9w=; b=VHOwXmdeEIQalt/ptA9tolcSE9sXB1xzhgFlGOLfzL9ZQMyhE7oM2Knb6+czOhhpZ8 +S3aN2G9voMgZFXgj1W76i5/Uv+D9Coete2zUr7TmDpbqu0ZFTVKgxvnIhbCSxYkJnKu w7/Eec2wz5l5c/d4KYe3KrV50zDQI9Br4KSI/JXSYaJCdfeOIkCMJJSmvjnc8FX2WpY6 HHmNSlwcE7Scyci+vE440QKx9hDStx4vEorUhO1VuMLvXFkcpC7rYn9gl6QUHC6DWD2J TNNX7v7Z2vgPyJ2bZQf5oWANsN0fuQ/FqwcQYew4CKuWJUq/dLKrNejqpr/Y91ZFJnMs WwWw== X-Gm-Message-State: APzg51D+LP+gsKauqtYuGVkna/Qp6796WC5mqSsRVkk+vZqiWWovOLqP y4cpzoCXye0TopC6Awrk5/ymmtS21QI= X-Google-Smtp-Source: ANB0VdaRv54f7pdgFOCYz1feR2nEWlZpbg7oBrt4fVJ4QHP+8CmbY9vK29df+pc7B0bgQsi+x+5RvA== X-Received: by 2002:a05:6000:10d0:: with SMTP id b16mr28803961wrx.226.1537361896373; Wed, 19 Sep 2018 05:58:16 -0700 (PDT) Received: from localhost ([2001:1be0:110d:fcfe:489f:80a9:5d59:c6bd]) by smtp.gmail.com with ESMTPSA id g17-v6sm3333006wmh.19.2018.09.19.05.58.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 19 Sep 2018 05:58:15 -0700 (PDT) From: Luca Boccassi To: dev@dpdk.org Cc: maxime.coquelin@redhat.com, tiwei.bie@intel.com, yongwang@vmware.com, 3chas3@gmail.com, bruce.richardson@intel.com, jianfeng.tan@intel.com, anatoly.burakov@intel.com, llouis@vmware.com, brussell@vyatta.att-mail.com Date: Wed, 19 Sep 2018 13:57:57 +0100 Message-Id: <20180919125757.17938-3-bluca@debian.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180919125757.17938-1-bluca@debian.org> References: <20180816135032.28283-1-bluca@debian.org> <20180919125757.17938-1-bluca@debian.org> Subject: [dpdk-dev] [PATCH v2 3/3] eal/linux: handle uio read failure in interrupt handler X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" If a device is unplugged while an interrupt is pending, the read call to the uio device to remove it from the poll wait list can fail resulting in it being continually polled forever. This change checks for the read failing and if so, unregisters the device as an interrupt source and causes the wait list to be rebuilt. This race has been reported and observed in production. Fixes: 0a45657a6794 ("pci: rework interrupt handling") Cc: stable@dpdk.org Signed-off-by: Brian Russell Signed-off-by: Luca Boccassi --- lib/librte_eal/linuxapp/eal/eal_interrupts.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/lib/librte_eal/linuxapp/eal/eal_interrupts.c b/lib/librte_eal/linuxapp/eal/eal_interrupts.c index 4076c6d6ca..34584db883 100644 --- a/lib/librte_eal/linuxapp/eal/eal_interrupts.c +++ b/lib/librte_eal/linuxapp/eal/eal_interrupts.c @@ -627,7 +627,7 @@ eal_intr_process_interrupts(struct epoll_event *events, int nfds) bool call = false; int n, bytes_read; struct rte_intr_source *src; - struct rte_intr_callback *cb; + struct rte_intr_callback *cb, *next; union rte_intr_read_buffer buf; struct rte_intr_callback active_cb; @@ -701,6 +701,23 @@ eal_intr_process_interrupts(struct epoll_event *events, int nfds) "descriptor %d: %s\n", events[n].data.fd, strerror(errno)); + /* + * The device is unplugged or buggy, remove + * it as an interrupt source and return to + * force the wait list to be rebuilt. + */ + rte_spinlock_lock(&intr_lock); + TAILQ_REMOVE(&intr_sources, src, next); + rte_spinlock_unlock(&intr_lock); + + for (cb = TAILQ_FIRST(&src->callbacks); cb; + cb = next) { + next = TAILQ_NEXT(cb, next); + TAILQ_REMOVE(&src->callbacks, cb, next); + free(cb); + } + free(src); + return -1; } else if (bytes_read == 0) RTE_LOG(ERR, EAL, "Read nothing from file " "descriptor %d\n", events[n].data.fd);