From patchwork Fri Jun 30 03:27:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rma Ma X-Patchwork-Id: 129129 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 6F3E442D65; Fri, 30 Jun 2023 05:27:19 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 0471E406B5; Fri, 30 Jun 2023 05:27:19 +0200 (CEST) Received: from APC01-PSA-obe.outbound.protection.outlook.com (mail-psaapc01on2067.outbound.protection.outlook.com [40.107.255.67]) by mails.dpdk.org (Postfix) with ESMTP id B28754021F for ; Fri, 30 Jun 2023 05:27:17 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=i0CRMhcixzCspl9g5QKH66eoWBDNQ754tvBdrsLjlEiS/fk+UDVZkJqPB3ShkxYVgVo8F6ExBrGgtkpxtSNHf/+cDCYmREi6bCSJnd4br6NUMGNFNcgxcqCUd9ckxMee8K50gaAG/vcZXboiLMuclr2bo5vVAq48927bZ3RRBXi3ueCeKlUgq3wgAm6H9UycSZ4bz8S5PaIFr+fMSvz4/SQ+vJECsJvhcQs/6k/MWuu8Vz0jiuJxU1hUDm+7ZeEpiYwAq9hszS+TM7hVzP1moA6YLw+yaA4jFyS52m1znyjZpVlgWs78T10LbrxsL+qhYpd/d59BQHEeY6guZF2fDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rGXT5dp55x53GCKMub9j80jYSeASZXKO0D8QUsjHpsQ=; b=Xz8e6XfeB9Kv+vhLQNLykampptTYgEf/ydOttqDIc1BGoZ1EJC4vv/mXK61JyKWp8RGMMzjGRckF0EgHtFqXWMdKz4Clix2+OFRBFo9xfeTIAPyMi6106cEqu/8fDOhQz0fQVsrmTBlK/h8TEBwCYVEWQ7Oudb0xQrmlrZqp0Tw83CrL6dPbfooWuPUwIlTiJ3Auzf30tQu4z25Oox+pnz48k/eTrnfvMPCirQULi0Bz2XyqfRw2Co8TzvpyJpFtk/pzd8s9X5q5s8kmos7N1QWf8xorKleuGhpEdcZ0VTHbm7iij7luyispkvQxuE/z+Gi4qHaAavvERdYiSbK1NA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=jaguarmicro.com; dmarc=pass action=none header.from=jaguarmicro.com; dkim=pass header.d=jaguarmicro.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jaguarmicro.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rGXT5dp55x53GCKMub9j80jYSeASZXKO0D8QUsjHpsQ=; b=QZvblBwpnRMk5wu8+M8404m3ky47isXivU+H3TQLEKNcrYmwhyNmX0EuHMmULBBwxv2+Sk6d8Y8HgO8B7KruRsAKiieqG3mmIoVePL2hUDdw0S5CqszBTmWv+r58sXzCWyHdVoqUbYxWgTo+IcrRqKO/Wf5yKLjDXjtOEfKsRgKTK9ILGaxG2Jn/yQMjBO7SLjz676I+NwRcm2gm4kF9ZAvnohu5lE0gQsi1Nf28VXhKI8UrTwdZXh0qYC4gnEs+jPNMFn/HXf28xCrHUNBUo1qLEis+nRtcjHtL6jEDDGKHEeUj2HRzK09cqZImMfOaM2Vbn93cbIPbChlNg3XASA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=jaguarmicro.com; Received: from SI2PR06MB4752.apcprd06.prod.outlook.com (2603:1096:4:14c::14) by TYZPR06MB4079.apcprd06.prod.outlook.com (2603:1096:400:7::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6544.19; Fri, 30 Jun 2023 03:27:14 +0000 Received: from SI2PR06MB4752.apcprd06.prod.outlook.com ([fe80::25f2:1ee4:9de:7b00]) by SI2PR06MB4752.apcprd06.prod.outlook.com ([fe80::25f2:1ee4:9de:7b00%3]) with mapi id 15.20.6521.026; Fri, 30 Jun 2023 03:27:13 +0000 From: Rma Ma To: dpdk-dev Cc: Maxime Coquelin , Chenbo Xia , Rma Ma Subject: [PATCH] vhost: add notify reply ops to avoid message deadlock Date: Fri, 30 Jun 2023 11:27:03 +0800 Message-Id: <20230630032703.1919-1-rma.ma@jaguarmicro.com> X-Mailer: git-send-email 2.17.1 X-ClientProxiedBy: SG2PR02CA0028.apcprd02.prod.outlook.com (2603:1096:3:18::16) To SI2PR06MB4752.apcprd06.prod.outlook.com (2603:1096:4:14c::14) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SI2PR06MB4752:EE_|TYZPR06MB4079:EE_ X-MS-Office365-Filtering-Correlation-Id: 50c53f68-a5d3-41df-e385-08db7919e5af X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F4iNaNxr33lJXBVULOhlW/k0JT5Zyw1P77Ovk1TD8AG0KWVtl/VlSaKd5m7aSMUaHK3F1lpF6rN2ac+sgv7g4+IGFg64BFibgfkFG2dm077xxBew4+FhvVoQ2U/n2BCb0RqEdUpOdA26fhIbroc8EV0/UZsPdeRc39YX5/TtRiT9NOeIZSj8A5AG9YIokCYOOAFAk/sWyRpNkv+xnotdFpkH2wCO+1vAAMfwDaqTDdO3pJ+z591G7t9rVApBc+mg/ZtWRAuGZRJsSgjza2JYzBoRCiTIhRhBuL+87rZwRD+jgRd/1x5VyFkR5OBseSy9iWDXZwJoNiFTfLzOlnnOdaSm5fZVQvjBExF8+Cuaa+1lWiJgt/LtH4/yNVArzHGLuNtrzUbaP1sWzZPxzAiIklA2pnChpnhMzmWKWUFr0a96qQKFrRrda8E/aRH2QjESEiTckOtvjj7CR1rQ1KkhBvbrFo1qGJGZfm7JnwiB6jsYkkE094T9EPy/v9L7TjbjCMgQ2hxAsgp0/Uz/pSY5NLJjuNtcXoS7Ag1v/pfMtc4QLLDZKroyKZ16pQfOl7nIGQvlcuPmaMi+v5fESnGC3eJgHf6ryNN1uYsH7ja/VpG8FM3lJfydqGpAe6CjiekW X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SI2PR06MB4752.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(376002)(346002)(366004)(136003)(39830400003)(396003)(451199021)(8936002)(8676002)(15650500001)(5660300002)(44832011)(66946007)(66556008)(66476007)(107886003)(6916009)(4326008)(2906002)(316002)(41300700001)(54906003)(478600001)(6666004)(6486002)(52116002)(6506007)(1076003)(83380400001)(36756003)(2616005)(6512007)(26005)(186003)(38100700002)(38350700002)(86362001); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Mi6w1dcRje0O9FqLx0altWSHbvzxAaLXSW6PvgDwwNe/GCGXhTx0VmjaNY7HvAt90kVgNkSiighs3g4awb4pUkrfHpjrtwiEmEhi+IkqF54BYpEf6hwQO1WQegp3OY7zexvyjfVTMGQYIbs6jjK+YlgdIMQHMN00z6XZjxbgSKoS53IDjImH3rTQ/qsN/upgPndp1Nex+dAuPSvjGdD3iuRlITHn+6zslcDb9Ifqq0Ipjl7L+pBXxcI8hsc8s55dQuPu4qABXHoSL/WAqDylWeVZPUALbBxCp7pK9hNWWEDyO4Irp1rWVXhkJuRqC1jHm+v5fCI7EIfoXHft0fIJPmnEpQmLYYLDkT6Zz7we2RlW7SdrzTcz4wKRcGEE2PEjUu1KIDL57KHGDJ26HLpX/nT9e5P/83lggHuqU6pgPY174A2Jrdu1kJFhVhm76b4yzBVSxNaEAR2SkuIwpmFRPFXHFNZ00ezywwyCuHwaIrUhpinmY7/J5ZnsSmBgbJUWmHAN5lhpWa1MOJiWKX1h0j83faeQpbjlSvYJteydarsY/u+MsPGkUK6TqU2YCSesPsMdT2dh/zHp6s+3qWQQzoEV9Ml/3QPaHD9Kan8HRC401MwYV6ndFzMIhMwFC4LnZveWqf+AWf6P2VTRECovfeEs0SdlRM+JX535N4tL/7OLz1GHqYh0V9ez0fy0fyPruaKpukYpt99jzM6ecp9OMfQi2+Sx9jEiNpq/0NvD1f9dDCNZc4CScsBRHwnUadF7KDHOmMSG6Heyap+jAYVcEyc/rKpWcaxXt1DWWbRVGGZU9KcgL7f17PFtMVaDf7sRcyTxMrC/2duxrGKooCK6U8aP+NK+qLNgm9ArjvGC2CtksoPTyGEGpozcYPIQ/x3SAU8FlUtWHDl+1qwXlp8aBxxyrCzB5Lgcb3xOqsKoWcKPLz58zJg7q571NoN4XKXXPDKyQTBMa0b7F3AmPr3o8+ZSP1yMfWGNd5UTzZk06z0B5Fs4DgrAuVhqxg+VIQHHTssUsh+NapNqlMfHNc9xmX2o7ATPr7FMa8U8NVPeBxyWJ2alho15dbu7oUIZ2LQAgMWcuW1Ij2oWVmvmPY+jjrKXUMFc+4IwfKmsGBx5cxiSD1jaG7SASA6NvFmcwzdAXy51n6+eCDayhKrXP3f+LxcPZuTXaq/usEwy9FbeK9DWPGqzWDySi0DeyVGvyEKYBHji4QdXOrJh+cRGMSURewQxA1MX1WDxuo0fLYY/EVNikXGkEZesPJ3RWCrAkadsPp7OyXZxrS9UXPDk9+9cvOa33cjD0JgVmlQqFinFquuhSGQ8041s4uR3e1/Hg4UHC7xv7U1MN9swJ/+h7zc7EpLqthXLTqaFwEheSSXLWBaqfITo1DSvHMgWQElsHo3KT+2ncLUadRb+O9lxgQzSwNzGonUvcaCzoJ6ggIZEfgJQnSmV8tryeXRtJnO+e6uAXZEiI0tNwO9u3DbGr5i99etGeNd9zZjxO4AF1wthGHXcanfYtNtINJXpvRCaZOZhCoshCLhx2tTpRFf9it4kWT5E6rAqpnSGvp80qHiTkON67M47F1MwqUit2nu2evwA X-OriginatorOrg: jaguarmicro.com X-MS-Exchange-CrossTenant-Network-Message-Id: 50c53f68-a5d3-41df-e385-08db7919e5af X-MS-Exchange-CrossTenant-AuthSource: SI2PR06MB4752.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2023 03:27:13.9121 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 1e45a5c2-d3e1-46b3-a0e6-c5ebf6d8ba7b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nSg4fBZEq7eW0hw1uG2bMfKRu1EnoenheM2q2Ts0hq8MsMq4S+wLouA24bo7ETV2fHIDpCaPxg1aRH6yN2eqTA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYZPR06MB4079 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Since backend and frontend message are synchronous in the same thread, there will be a probability of message deadlock. Consider each driver to determine whether to wait for response. Signed-off-by: Rma Ma --- lib/vhost/vdpa_driver.h | 3 +++ lib/vhost/vhost_user.c | 23 ++++++++++++++++++----- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/lib/vhost/vdpa_driver.h b/lib/vhost/vdpa_driver.h index 8db4ab9f4d..3d2ea3c90e 100644 --- a/lib/vhost/vdpa_driver.h +++ b/lib/vhost/vdpa_driver.h @@ -81,6 +81,9 @@ struct rte_vdpa_dev_ops { /** get device type: net device, blk device... */ int (*get_dev_type)(struct rte_vdpa_device *dev, uint32_t *type); + + /** Get the notify reply flag */ + int (*get_notify_reply_flag)(int vid, bool *need_reply); }; /** diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 901a80bbaa..aa61992939 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3365,13 +3365,14 @@ rte_vhost_backend_config_change(int vid, bool need_reply) static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, int index, int fd, uint64_t offset, - uint64_t size) + uint64_t size, + bool need_reply) { int ret; struct vhu_msg_context ctx = { .msg = { .request.backend = VHOST_USER_BACKEND_VRING_HOST_NOTIFIER_MSG, - .flags = VHOST_USER_VERSION | VHOST_USER_NEED_REPLY, + .flags = VHOST_USER_VERSION, .size = sizeof(ctx.msg.payload.area), .payload.area = { .u64 = index & VHOST_USER_VRING_IDX_MASK, @@ -3388,7 +3389,13 @@ static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, ctx.fd_num = 1; } - ret = send_vhost_backend_message_process_reply(dev, &ctx); + if (!need_reply) + ret = send_vhost_backend_message(dev, &ctx); + else { + ctx.msg.flags |= VHOST_USER_NEED_REPLY; + ret = send_vhost_backend_message_process_reply(dev, &ctx); + } + if (ret < 0) VHOST_LOG_CONFIG(dev->ifname, ERR, "failed to set host notifier (%d)\n", ret); @@ -3402,6 +3409,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) int vfio_device_fd, ret = 0; uint64_t offset, size; unsigned int i, q_start, q_last; + bool need_reply; dev = get_device(vid); if (!dev) @@ -3440,6 +3448,11 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) if (vfio_device_fd < 0) return -ENOTSUP; + if (vdpa_dev->ops->get_notify_reply_flag == NULL) + need_reply = true; + else + vdpa_dev->ops->get_notify_reply_flag(vid, &need_reply); + if (enable) { for (i = q_start; i <= q_last; i++) { if (vdpa_dev->ops->get_notify_area(vid, i, &offset, @@ -3449,7 +3462,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) } if (vhost_user_backend_set_vring_host_notifier(dev, i, - vfio_device_fd, offset, size) < 0) { + vfio_device_fd, offset, size, need_reply) < 0) { ret = -EFAULT; goto disable; } @@ -3458,7 +3471,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) disable: for (i = q_start; i <= q_last; i++) { vhost_user_backend_set_vring_host_notifier(dev, i, -1, - 0, 0); + 0, 0, need_reply); } }