From patchwork Tue Jul 11 09:25:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rma Ma X-Patchwork-Id: 129439 X-Patchwork-Delegate: maxime.coquelin@redhat.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id B92AB42E44; Tue, 11 Jul 2023 11:25:31 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 3471A40A7D; Tue, 11 Jul 2023 11:25:31 +0200 (CEST) Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2088.outbound.protection.outlook.com [40.107.117.88]) by mails.dpdk.org (Postfix) with ESMTP id C42BF4003C for ; Tue, 11 Jul 2023 11:25:28 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UNbzLoUsvPrAqf3IarkgeFacK6Zpd1qFy0kJ3v1cZdJm4kGFDR7qdTeOT6QlzmZzxvx1mxNf1p2YGnk6Zr8Ty2dbt3WnolHVG9QUjWwINP/PLiYYF9dGlS2M/MnVitkkfL8ET/qt2QbCOOwiCCLHBDcQlzLSr1WKUZRdr6ydjDjsvikQamLhQ3gApq9FnCvCiqa6rCg88CGCKhTdhiuDjiG10IdwcQ6WnQ2nwWUTFOCoZEOXLBpLODMtZMFfRQ0mOSKJsqeQWSPNqbF/ION1cqcwA/XsfCQ62gc+PKjQMEYrJ4hubjQnS/AWQ+eXUumUIAxSnQoni3/+khNQkWltpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YMCbX7hBQIh9rpNOQpOh+ovd1CA9QKnw6k7VnkP7rrQ=; b=NkrFffZpCybnAJTk8aePTgH5OYFwr6xpmDfrBLV90FKOcX+c1THB1wbooz9NQ5bDBWYV66xhZWpNqK5+0G3x1Qq+uWEFz6GWoDpi2QT4edJKTymwAVroMaTgzjv7XmPFIjHZPC1cIbWBBgKhgIrrvHidgnVISaAWp6Jwf5/rjQo7pJ0JUeCx9a8yGOMFVF2tMo6hnjcXYogH4PmqZ871Yo341kI1MTXkzTVZwzKUPb2lyauDYQ6lFZjs3BMVyQV+J18v+u4QJNH36egkk1pQVb0EVyp6v8swDKxdBB2+wA7Rjp/XrT2nN2osxotzVnTZJWFxCg/Po7n59ugKe3Uatw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=jaguarmicro.com; dmarc=pass action=none header.from=jaguarmicro.com; dkim=pass header.d=jaguarmicro.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jaguarmicro.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YMCbX7hBQIh9rpNOQpOh+ovd1CA9QKnw6k7VnkP7rrQ=; b=m0CTeaQj/ScYQX3Ki83QOsraPx8j+SyKihUPbYvUPp187ZMZvN8IKtElgL5d9VEtvZgAWcn33Jvj7/30EMPrjPTMrVNvRUJw4eKmfmKtJb9ygmP++7cpz1ONa0ss12E6BjxeWcVPZq6zjShpPsciQCVG2QsSxU5W6y8ID/Q/4nh0O1P/JEtN9Wp0w1HyqNoeRZUB5Y7VXKdyJOxf3Ta2j3JBfeli1D7hBjDw9axrA3BOgpAbDWgZxMBLZQ0PLkvGO4fxKUnYrAT0jYkfYNL3yolSHKwyIC/bKNpLZ0f7PcyZRNVnyFuXoG+TIHY27ET12ZlpYYZILiA7k0TL8C9ZXQ== Received: from PUZPR06MB4742.apcprd06.prod.outlook.com (2603:1096:301:b5::13) by TYZPR06MB5949.apcprd06.prod.outlook.com (2603:1096:400:337::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6565.17; Tue, 11 Jul 2023 09:25:25 +0000 Received: from PUZPR06MB4742.apcprd06.prod.outlook.com ([fe80::936c:12e8:b8ef:7d51]) by PUZPR06MB4742.apcprd06.prod.outlook.com ([fe80::936c:12e8:b8ef:7d51%7]) with mapi id 15.20.6565.028; Tue, 11 Jul 2023 09:25:25 +0000 From: Rma Ma To: dpdk-dev CC: Maxime Coquelin , Chenbo Xia Subject: =?eucgb2312_cn?b?u9i4tDogW1BBVENIIHYzXSB2aG9zdDogYWRkIG5vdGlmeSByZXBseSBv?= =?eucgb2312_cn?b?cHMgdG8gZml4IG1lc3NhZ2UgZGVhZGxvY2s=?= Thread-Topic: [PATCH v3] vhost: add notify reply ops to fix message deadlock Thread-Index: AQHZriKIUaeiph0Px0C7vwJ57C7k96+0VosO Date: Tue, 11 Jul 2023 09:25:25 +0000 Message-ID: References: <20230704023228.24879-1-rma.ma@jaguarmicro.com> <20230704025157.26024-1-rma.ma@jaguarmicro.com> In-Reply-To: <20230704025157.26024-1-rma.ma@jaguarmicro.com> Accept-Language: zh-CN, en-AS, en-US Content-Language: zh-CN X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=jaguarmicro.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PUZPR06MB4742:EE_|TYZPR06MB5949:EE_ x-ms-office365-filtering-correlation-id: a47b2848-c1d4-4efc-9ea2-08db81f0c258 x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: dDTbe0hoolfHkolwechGULphh+6KQ2xMLW3QZgixkAhbuf2cgOOgay2o4nL7UzSkVfL38yu8OeoqS6Vb1kzGmCDBql0oBbNVdMmJe9G1E9WehHhraCH7I5w5xrTk6gVUF+CppAjO3gt+UeOkIBzEdUEo8HKOptbEwW7Q7ij+uTDPC4m4XkA6XHE2go1lfejbhhCWAj3fWZNPxg157uRGM8QjtvB5bNpiMywNl53+whZQCM+FAutzB0b0dSc1T3epw3735y7GrRtFluP/n82iSGo6UPev4jr1RPKghdWrRtQ3RUfsCQ56+f1VM2PW5kdQM3OYFTnMx2V69g5IbfLksmFIcTSE54Q7Lz3nEtjhtPDRtuOiI6KYQHIEh2ryJFA69jxnOSVwYRb4XTVSqKnSwe4Og4ymHhT4JVzsLO5HSG0u5X2FTTDUAYyvOs5BxaxWqhsBqrtiKtQk6dL4xwJb1Deh4VAG982dTUPLd7EHLOZFPJMculM7tiY0SBKXVJTt5YdxB87zLaRhYrmWTCDVj9sZA4LnPQtmXD7v9/JNboB3oHZjb4qZ50p+JE8p+dpG/gTa5H6EyB7VgPBCozbDEUx7N5AjX7cw5gjSAoKBy6HxISqbJm2F4zoX8NzkcVQX x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PUZPR06MB4742.apcprd06.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230028)(346002)(376002)(39830400003)(136003)(396003)(366004)(451199021)(55016003)(54906003)(122000001)(66476007)(71200400001)(66946007)(91956017)(478600001)(6916009)(7696005)(8936002)(4326008)(76116006)(38100700002)(41300700001)(66556008)(186003)(66446008)(9686003)(83380400001)(6506007)(26005)(316002)(5660300002)(33656002)(15650500001)(2906002)(86362001)(52536014)(38070700005)(44832011)(64756008)(224303003)(19627405001); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?eucgb2312_cn?b?S1Z0eGVKMXgvSTBpb0g2T3Iy?= =?eucgb2312_cn?b?NDBaNEh4VkUzU3F3ZlBrd3FPVU9VVXZReUJveDZPREVaUmN4ZEVPRU1STE1WZEZq?= =?eucgb2312_cn?b?YXRHNXRpRXNQRmtnRkY0elpRUVEwb05WeXkydWRoT21nbTloNmZZTmZ2ZHFBTTlB?= =?eucgb2312_cn?b?bVRZYlRXeVRtTmUxNDVpek5UK3pwNVczTEJoS1g4dFlxWkROa1Q5SEFRUTE1WEpK?= =?eucgb2312_cn?b?MXVyVENJUGJ1Z3ZZSks4RFJEOXJhZ2NIZ2V1OFdhSGZyZ2QvZzBGVjJGZGM3Wlhv?= =?eucgb2312_cn?b?TUQ1N3RpR1NNZmdIMCtPZDloMFZSYytTdm8yeHc0c1FaWGNnRDRpM3pKOG5NRU9C?= =?eucgb2312_cn?b?eVl2dU1ETE5aQS84K29WM2l6dmhMeXFkMzdpblN6QkdzTWdZbHlEWThzbWZuMTVh?= =?eucgb2312_cn?b?ZDFEbGh6MUc1Yjk1aEpTWU1uYitpTVB3NWg3SGEzSDRLUkhWdXppZkltd1JONlRo?= =?eucgb2312_cn?b?TG1wN3NHK2dCQnNxQWx5bUMxMnRETTU3L0l5VUNjaE1rRzlsRENqREowMVRtcUJ4?= =?eucgb2312_cn?b?TklGQ1E1UU1zTVdlaDZQd0tMUTZJSDRFcy91MXJGVG43YjJqYlpMSWYvcFFFN2Fv?= =?eucgb2312_cn?b?UUgxMllZTHlHcFI1SWw3Z2FWZUVuTTUrcUQvaGwySklCMmdvZmJiQmRNY3VjNnE0?= =?eucgb2312_cn?b?emlSdFBqWjJsY2hZVHVOK2YvK1p3MXpOTC9GUENJN2crOVNUSXA0TlMyRHNoUlhN?= =?eucgb2312_cn?b?SWVhR2l0MVQ3N1lRVEhHQU9HeHZWR1dLZEp6Mzhma2h4Mk00cUhqb0ZtWjRLdHNK?= =?eucgb2312_cn?b?MFBRa0RlZlE2dlc2MVFUWkxucHNHNTBkbFhMaEhyMG1OdGcrbVR3NG9rWThENy82?= =?eucgb2312_cn?b?NktsMlI1UTUvRUVxbXRqQkhaRDN0TTZ1Rnpjd1AzWFpFRmhjaVk4ZnBkdXB2UXVp?= =?eucgb2312_cn?b?bnNoWjNCNnhjSTJvUlg3MGJKNEQyYmdhbXJodjFqUWJyam9qczcwY0I3QThFRmtH?= =?eucgb2312_cn?b?M21JU0tlRU56Yldiam5XNXFOT3d3N0JDZVVNS2dxOWJsVFlNZm4xQ2IrNjlyUTJu?= =?eucgb2312_cn?b?UzRmTk5NcEJJOUdUYndnZHlkUlFjUi9tTHlRZytyUWpIU0VlNEk3Mk1pTnVhTkJG?= =?eucgb2312_cn?b?b1FNV0lSbkx3MTBqRGcxVWsycVZjVnU2Qi9LZGtJWGVPT0dRaEE2KzRDU2FSRlli?= =?eucgb2312_cn?b?R2tFK0hpV2ljbXRQbWMxdlJnUlFjTTVTYkdzckVBWTFnd283SEtmTElQTlUvdmtB?= =?eucgb2312_cn?b?cWNiN1lwRHppczRCL282cmhXM2t3d3pXbjM3UVF1NXlSc3RHaktYYk84V1FBSkZi?= =?eucgb2312_cn?b?QzRIaEpZMFZHbHpEVHlNbityVVB6SUZjQzZyRFNwYmhkNDFxSzFGR1NiNzZaYWFV?= =?eucgb2312_cn?b?bm51cnBHYWdxN1o2YVVBblZ4UVRUeVBoa1NwU1N3RFF4bTM0SjFIZDkvVk5Wc0pk?= =?eucgb2312_cn?b?cWprcHduMFRXWWRnSEdROWxQUTdpaTEwWkRCeTc3ekl5S2VTSC9jWnQrblplNms1?= =?eucgb2312_cn?b?WUNUdnBXelYvS3IrYUI0Q25JY0hBblFmei9pZEdPaEs0VDY4S1A3blBMVjFuV1Vm?= =?eucgb2312_cn?b?ZWpRTHNTbUtweTg3S2NESVZ1eVpjdHgzYXBPVWxEUTlBalJBaFJWZTBRaWIxUitW?= =?eucgb2312_cn?b?R2dHK2Fyd09La2lKdHdiKzQ5cmJJTjhXbklMUmsyRC9OSWZha0pOSGpoUmRGNStj?= =?eucgb2312_cn?b?VVUxd3Fud3lRU3ROVmFDMTFFLyt3a0RRak9BTE12UjVtYkJnbVBpYW1zWDFMZE4w?= =?eucgb2312_cn?b?NGdFbUEySXY4T1Vya2k1Y1FzSmhQbk1KbmVHSUZKZlp4aGhQelRsN0dka0h2b3pr?= =?eucgb2312_cn?b?NWtESFNyVVFwVjR2d0NEZ08wUmh3NGZtOWZYaWtCQXlTRzFoRXNHNkVUNWxpRGE2?= =?eucgb2312_cn?b?UnA0OVRhRnV0Y0RSSFZ1QTJnRmRsSVY3NDhTQU1sekRuSll4K2lSbDB2RTc2QUc0?= =?eucgb2312_cn?b?eW9aeTZxWjNscXI4TEtXbUVmKzIzYXpzay9TdEtGeFJ1dWpZd2xUOThWYkpsaWR1?= =?eucgb2312_cn?b?d0tZZllOeW43ZTFqdGlhS20zNnp4Q2htOGtrUEx1MGs0NUxMZ1YwckpJalp2M2Fr?= =?eucgb2312_cn?b?NlEyZXFmY0tmTEcwQmNGbE9DZzZPL0E5TT0=?= MIME-Version: 1.0 X-OriginatorOrg: jaguarmicro.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PUZPR06MB4742.apcprd06.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: a47b2848-c1d4-4efc-9ea2-08db81f0c258 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Jul 2023 09:25:25.5252 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 1e45a5c2-d3e1-46b3-a0e6-c5ebf6d8ba7b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: j/FkDYxtNm8bFceesXyPdagnf1M0NJYlzgQZI8kP81LFj0BDHRWNERapONVbOM5uepRKHauKytNYSTvDckDxZA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYZPR06MB5949 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org > Since backend and frontend message are synchronous in the same thread, > there will be a probability of message deadlock. > Consider each driver to determine whether to wait for response. > > Fixes: d90cf7d111ac ("vhost: support host notifier") > Cc: maxime.coquelin@redhat.com > Signed-off-by: Rma Ma > --- > v2 - fix format error in commit message > v3 - add --in-reply-to > --- Hi Maxime, This patch helps to fix vhost-user message deadlock, could you help review it? Thanks. Best wishes, Rma diff --git a/lib/vhost/vdpa_driver.h b/lib/vhost/vdpa_driver.h index 8db4ab9f4d..3d2ea3c90e 100644 --- a/lib/vhost/vdpa_driver.h +++ b/lib/vhost/vdpa_driver.h @@ -81,6 +81,9 @@ struct rte_vdpa_dev_ops { /** get device type: net device, blk device... */ int (*get_dev_type)(struct rte_vdpa_device *dev, uint32_t *type); + + /** Get the notify reply flag */ + int (*get_notify_reply_flag)(int vid, bool *need_reply); }; /** diff --git a/lib/vhost/vhost_user.c b/lib/vhost/vhost_user.c index 901a80bbaa..aa61992939 100644 --- a/lib/vhost/vhost_user.c +++ b/lib/vhost/vhost_user.c @@ -3365,13 +3365,14 @@ rte_vhost_backend_config_change(int vid, bool need_reply) static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, int index, int fd, uint64_t offset, - uint64_t size) + uint64_t size, + bool need_reply) { int ret; struct vhu_msg_context ctx = { .msg = { .request.backend = VHOST_USER_BACKEND_VRING_HOST_NOTIFIER_MSG, - .flags = VHOST_USER_VERSION | VHOST_USER_NEED_REPLY, + .flags = VHOST_USER_VERSION, .size = sizeof(ctx.msg.payload.area), .payload.area = { .u64 = index & VHOST_USER_VRING_IDX_MASK, @@ -3388,7 +3389,13 @@ static int vhost_user_backend_set_vring_host_notifier(struct virtio_net *dev, ctx.fd_num = 1; } - ret = send_vhost_backend_message_process_reply(dev, &ctx); + if (!need_reply) + ret = send_vhost_backend_message(dev, &ctx); + else { + ctx.msg.flags |= VHOST_USER_NEED_REPLY; + ret = send_vhost_backend_message_process_reply(dev, &ctx); + } + if (ret < 0) VHOST_LOG_CONFIG(dev->ifname, ERR, "failed to set host notifier (%d)\n", ret); @@ -3402,6 +3409,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) int vfio_device_fd, ret = 0; uint64_t offset, size; unsigned int i, q_start, q_last; + bool need_reply; dev = get_device(vid); if (!dev) @@ -3440,6 +3448,11 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) if (vfio_device_fd < 0) return -ENOTSUP; + if (vdpa_dev->ops->get_notify_reply_flag == NULL) + need_reply = true; + else + vdpa_dev->ops->get_notify_reply_flag(vid, &need_reply); + if (enable) { for (i = q_start; i <= q_last; i++) { if (vdpa_dev->ops->get_notify_area(vid, i, &offset, @@ -3449,7 +3462,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) } if (vhost_user_backend_set_vring_host_notifier(dev, i, - vfio_device_fd, offset, size) < 0) { + vfio_device_fd, offset, size, need_reply) < 0) { ret = -EFAULT; goto disable; } @@ -3458,7 +3471,7 @@ int rte_vhost_host_notifier_ctrl(int vid, uint16_t qid, bool enable) disable: for (i = q_start; i <= q_last; i++) { vhost_user_backend_set_vring_host_notifier(dev, i, -1, - 0, 0); + 0, 0, need_reply); } }