From patchwork Tue Dec 26 02:37:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chaoyong He X-Patchwork-Id: 135566 X-Patchwork-Delegate: ferruh.yigit@amd.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 5D1204378F; Tue, 26 Dec 2023 03:39:39 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A8F0E40A6C; Tue, 26 Dec 2023 03:38:35 +0100 (CET) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2100.outbound.protection.outlook.com [40.107.236.100]) by mails.dpdk.org (Postfix) with ESMTP id D793240A73 for ; Tue, 26 Dec 2023 03:38:33 +0100 (CET) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=abNrd//aHHWNDrazOFfuQBR/krTRgOAdxoIBZnuNKVoKIufoW0llAeiPBXyKNyhAgpBp85NGWoUV4h31Gr+l9G1uyMd/lA1ztxGRhfZmgnyEfIjdWFHReL4tfGP7T8gzIfq75Zv7Ls+CF6upX/pK1IjsA+ndMorJrPWduGRN8bK632N6BGet/JLGT+nx/zzCTsrrRkjj/NZ/s0iopzWhPgEO9EjM6urZnh/8HP88/K6tOHUQaNHsVAzxG6bcG+LqSrE4OTWiRsCVQmT07PwvblpgBPIbdifqCpzevttepSKdi0IXNdX9bjEtCszEy7q2iSn/9v2DzEYG1tQWCUMRkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LP2kt1iTKA9LRto/Lv+MJPqkg+wcZDxliYzWTtLn/k4=; b=kS3dge0Eo0+4vah8y21KuJakxH2QBtViRUenXy402bATLhV5KiB1NQto7pC06JasVaDvvRScRIq/i2WMJ+X6XFqQokCXosXY8NI7wqLcZEBE0KD2GgPUJX6aSPZB+FKeI9cioaBer4/N0Iof4nAYgtKn6CMkCGpHzP/Njmf9uJ935gOI6P6z7nNqh0I8DPVjo8Afh7VPVnCRrdc7GfbDq5ybGm+amQbgfB6vCAch2IjugQfxHXB87pYkGI5efS1A/7IgBJQi7A9hMSNOCBRTlXq8vK84mV+iGNIlRd7EGv5DZk/Oo1MZyPbK5I4NFmvnrsRBFDFzESHxz1KbT1OpXQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=corigine.com; dmarc=pass action=none header.from=corigine.com; dkim=pass header.d=corigine.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=corigine.onmicrosoft.com; s=selector2-corigine-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LP2kt1iTKA9LRto/Lv+MJPqkg+wcZDxliYzWTtLn/k4=; b=FHPWC/hUm5PY/nMvsp0fO16iirVvDkSqW6kNXTxEKe3alqWJn/qJsRNRhgr7lDwaA2thU8l24C3/uCR3BMUNnHPAYigO2Gjtr4ofZ+O2Pb4yK32vYTfGk/UbO+kNhwavTudhvEXPJZKbXv4VoqdOWVTdpzLN83rgvGxXFw3oLeI= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=corigine.com; Received: from SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) by MW4PR13MB5864.namprd13.prod.outlook.com (2603:10b6:303:1b7::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.26; Tue, 26 Dec 2023 02:38:31 +0000 Received: from SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b]) by SJ0PR13MB5545.namprd13.prod.outlook.com ([fe80::45b3:d83c:cff2:4a1b%6]) with mapi id 15.20.7113.026; Tue, 26 Dec 2023 02:38:31 +0000 From: Chaoyong He To: dev@dpdk.org Cc: oss-drivers@corigine.com, Long Wu , Peng Zhang , Chaoyong He Subject: [PATCH v5 11/14] net/nfp: support bond member notification Date: Tue, 26 Dec 2023 10:37:42 +0800 Message-Id: <20231226023745.3144143-12-chaoyong.he@corigine.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20231226023745.3144143-1-chaoyong.he@corigine.com> References: <20231018074821.1908906-1-chaoyong.he@corigine.com> <20231226023745.3144143-1-chaoyong.he@corigine.com> X-ClientProxiedBy: SG2PR01CA0156.apcprd01.prod.exchangelabs.com (2603:1096:4:8f::36) To SJ0PR13MB5545.namprd13.prod.outlook.com (2603:10b6:a03:424::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR13MB5545:EE_|MW4PR13MB5864:EE_ X-MS-Office365-Filtering-Correlation-Id: 2dcc6c73-aedb-40b3-f4a5-08dc05bbbfb7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3e0khEzwbbNa1NOIeb3e9sU3J0OJ93Ea7z8CJP4FmwCNVEmeSyHmuLkhWFBGjT7VHvxlmlmomYrIVQLpVqKO1tMUSYIEXSWxGsT/G5+N7j98HAFwspNs6eQ4ZOSI81tUDavysyv/ah8Nl0P7JeJoW03mC51YKN6s8ZeMqDEA3Cxw/xKDMt17tK3GiCDdrTz5TWOFT1ChKxvHb0hzQJ4P9S4IvcCEHEPKU7/FRK6E638A2i5t75y5WFatXy9tu/KedeQt7etZeedW/QzFVcxGFLHHmST6rxSgp881Zd3lEWne5Nn6oIBnd8DyhcW1ikdBF/19TZxrdfQBWAfu19+RoFdMKV0FfwVEFzuqfw9dudTr4yX3BDhWwQ8341VUW5wnFsf/Jc5IUoF9PpX6b+g60Z3JiK91jnpZ4xfEpvlMtlGMtA2QdS0jO7uxz3OqUNHJ294iX0pBybGQVZCBy3Om/b/UzGvasYsoadckrOJff8JjCmjUB8JthhT4fnW2mjsLI63uGT/IfK8CTDllCnBnYGEpxaqx4J4EHm+7TNYfT8LL1/dcB6kxo39MV2Cr3eshy2+eXmh9idY8imCvqSunew9nDDhtbNwuQ4Ac717++yn/ma7ghG75xoB1AvarJ5v4PZuDiZqMFUawdb7luklVvp2NA5iw6d/gJ3xVErc+61TkGveFL7rf4CDAGbkQygL9wMuNR+Vp7n3TDR+V6E2/dA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SJ0PR13MB5545.namprd13.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230031)(39840400004)(376002)(396003)(366004)(136003)(346002)(230273577357003)(230922051799003)(230173577357003)(451199024)(1800799012)(64100799003)(186009)(83380400001)(41300700001)(2616005)(38100700002)(26005)(107886003)(1076003)(316002)(4326008)(54906003)(8676002)(2906002)(5660300002)(15650500001)(44832011)(8936002)(478600001)(6512007)(6666004)(6506007)(52116002)(66476007)(66556008)(66946007)(6486002)(6916009)(38350700005)(86362001)(36756003); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: t+52WF+8aq6Ke2AhAMsoLbnnMs2CZQVTXk6oABxvxMg7fxYCJ6RFjS20vEc4vkyBQIrm253ik8klbfuDO/UBF+POyarAVgeWm45Zn0dyhU2xW7nU0jqDFDljHJwEQ4XCJ26VPoNTU1VoLp78I1alT2mMpPfnoRfOiGm88leibPJvvXJZK1MXQCypJ3d0OzB9FMmNmbDpt0pc6xdAO3HVweYeGsrg0uwtytu7T4ev9gfg7FcA5yH3oxHFbIf2odxbJl9dfWEkhZMOrPWOj4yra+f/RiBT2TasbAosnIhli5RWTrZMCIHNMP+G9foX7by4Gm0Bfm1eQp2Rn5l0kNkQru5gRb9iPryq/duH+adM4yldkU59tdf4+/ig9RJwm5l4VXQ4Td2Xf0YfcayY0qyPnA+xf7sxTSH5Xi5kJ1bRZMKDMhzElXY9q9uipBUp/dCEOw+9ito+DcIQUjIwPDQcpMlz1Xl86w9D8ut5NVJ+rdQ6927q5G9igs4Zql2P6QyWQds29TNo7VH2xn21J4FSinWtmiOaCdp3Tx+3UYdcweCN1dfweeA6ub/+PReWHUH83eilEaAePru+wiFW8Q/+X29n99ZUo9T2bY21XdIEeyDuyZBSqE0YTRPQHnEaY2u4P21I5v4NCVNqOsibZSv17uf6On0/dugg1xzG1aCUNAEz/lVd8CVqSKMLt0jynvSBL4VrlX0+rO8QPR2CBNj/JGuC6X4/8aKC0LxTDlb44yijb1gLJZBMB0v/V2lrXLnRX4vtNHM+Y6ig9cYQ1vleGuXEciAcCEdOHfl4q0lXEl0Xzu9vgn3QSiv/4BUw5TXyp+XWKlP/NIvBu8jvtFKJFDxGgLViUPDbLc5PLlAchs213UsmFwuK+WPqwzeB+eIrVM9K+F+czrFojWUNbUO/3ocZtCFgLLce3eMEUz3+MjofuDw0mZQiZuuuNk4MBDQn19u6VRulMbKr9z21hGdzbOlDdVCTx3rNn5j6fv23UearUtyTSeJftnY0SPS9y6cV3/p7YJ5C/XrdPWczmuh/5A240y/yMpbmZAmuQdzugoGmXy5grIi3PFyol+Ftw0eg2JOqdzwBsFnrmRc9SvKfl27w26yIO47XUYLBAieYX6dKBpJ1iYzBNJge3RpBeq2MBjpw/BJnqnog7YjLyzzBmhzZYwhtY3/g2+YQH40tpMqQNVwDb6WAMbMomBHt20rn100104hGwPN55x2gJ4ulc5NwHMM73141HPiuQ09+i3Gc+XSbHaBAMNn/XrsAM+mp1iy+rX5ePuY62M94xzlYHvdyatLSmPdXlkEgvp9NfvIFT0486Cfw30Jg0y2cWMqKwwgHdSMOyJ9mNRkI5blv+aY/iA84NQjgtT8CmyJ2uEZFXX+lJMm2Q9I0x8CPoVTPXmGj+N9DUBgmfWNR8fXDcmB2ZlgA54y8TtxiHFj4kGnLVrTlIoByEo6UbXCyUzRefuM/b/rJW/vB9dngvbmDaaYwfW5rJTPYWYNCZa7U1Ua0iW0OD9S4ixQuJ9aVkh5ztK+273kxQhmsdWlmXG0orndyI05KhYhrFhp2jdwUNWekGYPI+FO5a/eGIJgCq6VGaotu3v1b3SR9C6uy5+SF3A== X-OriginatorOrg: corigine.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2dcc6c73-aedb-40b3-f4a5-08dc05bbbfb7 X-MS-Exchange-CrossTenant-AuthSource: SJ0PR13MB5545.namprd13.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Dec 2023 02:38:31.4884 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: fe128f2c-073b-4c20-818e-7246a585940c X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /aLytOgRYFE3zwCh2GxDY4kaQluh+J8AtI/GRmA/6oZD2gIpuBD9GS8pHHwUQjh4mVvmHpOqmVbQQgkoF9QLBCLixpbrjOUL+iAAeACWRrE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR13MB5864 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org From: Long Wu A NFP representor port can receive a bond notification. Then driver will parse this notification into one of these two events: 1. Bonding port configuration may have changed. This includes creation of a bonding port, removal/addition of a member port, changing the bond mode, etc. 2. Bonding port is deleted. Signed-off-by: Long Wu Reviewed-by: Peng Zhang Reviewed-by: Chaoyong He --- drivers/net/nfp/flower/nfp_flower_bond.c | 183 +++++++++++++++++- drivers/net/nfp/flower/nfp_flower_bond.h | 2 + .../net/nfp/flower/nfp_flower_representor.c | 21 ++ 3 files changed, 205 insertions(+), 1 deletion(-) diff --git a/drivers/net/nfp/flower/nfp_flower_bond.c b/drivers/net/nfp/flower/nfp_flower_bond.c index c814c0d4ea..523e0025ad 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.c +++ b/drivers/net/nfp/flower/nfp_flower_bond.c @@ -46,6 +46,39 @@ nfp_flower_bond_increment_version(struct nfp_flower_bond *nfp_bond) nfp_bond->batch_ver += 2; } +static void +nfp_flower_bond_group_id_clear(struct nfp_flower_bond *nfp_bond, + uint32_t id_clear) +{ + bool *group_id_map = nfp_bond->group_id_map; + + if (id_clear >= NFP_FL_BOND_GROUP_MAX || id_clear < NFP_FL_BOND_GROUP_MIN) { + PMD_DRV_LOG(ERR, "Try to clear invalid group id %u.", id_clear); + return; + } + + if (group_id_map[id_clear]) + group_id_map[id_clear] = false; +} + +static int +nfp_flower_bond_group_id_get(struct nfp_flower_bond *nfp_bond, + uint32_t *id_ret) +{ + uint32_t id; + bool *group_id_map = nfp_bond->group_id_map; + + for (id = NFP_FL_BOND_GROUP_MIN; id < NFP_FL_BOND_GROUP_MAX; id++) { + if (!group_id_map[id]) { + group_id_map[id] = true; + *id_ret = id; + return 0; + } + } + + return -ENOSPC; +} + static enum nfp_flower_bond_batch nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, struct nfp_bond_group *entry, @@ -65,6 +98,7 @@ nfp_flower_bond_remove_node(struct nfp_flower_bond *nfp_bond, entry->offloaded = false; if (entry->to_destroy) { + nfp_flower_bond_group_id_clear(nfp_bond, entry->group_id); LIST_REMOVE(entry, next); rte_free(entry); } @@ -139,6 +173,7 @@ nfp_flower_bond_cleanup(struct nfp_app_fw_flower *app_fw_flower) pthread_mutex_lock(&nfp_bond->mutex); LIST_FOREACH(entry, &nfp_bond->group_list, next) { + nfp_flower_bond_group_id_clear(nfp_bond, entry->group_id); LIST_REMOVE(entry, next); rte_free(entry); } @@ -271,16 +306,162 @@ nfp_flower_bond_change_linkstatus_event(struct nfp_flower_bond *nfp_bond, *bond_port_flags |= NFP_FL_BOND_PORT_CHANGED; } +static struct nfp_bond_group * +nfp_flower_bond_group_create(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + uint32_t id; + unsigned int numa_node; + struct nfp_bond_group *group = NULL; + + if (nfp_flower_bond_group_id_get(nfp_bond, &id) < 0) + return NULL; + + numa_node = rte_socket_id(); + + group = rte_zmalloc_socket(NULL, sizeof(struct nfp_bond_group), + RTE_CACHE_LINE_SIZE, numa_node); + if (group == NULL) { + PMD_DRV_LOG(ERR, "Unable malloc memory for nfp bond group"); + nfp_flower_bond_group_id_clear(nfp_bond, id); + return NULL; + } + + group->group_id = id; + group->main_dev = bond_dev; + group->dirty = true; + group->offloaded = false; + group->to_remove = false; + group->to_destroy = false; + group->member_cnt = 0; + group->group_inst = ++nfp_bond->global_inst; + LIST_INSERT_HEAD(&nfp_bond->group_list, group, next); + + return group; +} + +static int +nfp_flower_bond_changeupper_event(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + uint32_t i; + uint16_t port_id; + bool can_offload = true; + uint16_t nfp_member_count; + struct rte_eth_dev *eth_dev; + struct nfp_bond_group *group; + struct bond_dev_private *internals; + struct nfp_flower_representor *repr; + struct nfp_app_fw_flower *app_flower; + + internals = bond_dev->data->dev_private; + app_flower = nfp_bond->app_fw_flower; + nfp_member_count = 0; + + for (i = 0; i < internals->member_count; i++) { + port_id = internals->members[i].port_id; + eth_dev = &rte_eth_devices[port_id]; + if (!nfp_flower_is_phy_repr(eth_dev)) { + can_offload = false; + break; + } + + repr = eth_dev->data->dev_private; + + if (repr->app_fw_flower != app_flower) { + can_offload = false; + break; + } + + if (internals->current_primary_port == port_id || + internals->mode != BONDING_MODE_ACTIVE_BACKUP) + nfp_member_count++; + } + + if (internals != NULL && + internals->mode != BONDING_MODE_ACTIVE_BACKUP && + ((internals->mode != BONDING_MODE_BALANCE && + internals->mode != BONDING_MODE_8023AD) || + internals->balance_xmit_policy != BALANCE_XMIT_POLICY_LAYER34)) { + can_offload = false; + PMD_DRV_LOG(WARNING, "Unable to offload mode %u hash %u.", + internals->mode, + internals->balance_xmit_policy); + } + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (nfp_member_count == 0 || !can_offload) { + if (group != NULL && group->offloaded) + /* Set remove flag */ + group->to_remove = true; + + pthread_mutex_unlock(&nfp_bond->mutex); + return 0; + } + + if (group == NULL) { + group = nfp_flower_bond_group_create(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + return -1; + } + } + + group->dirty = true; + group->member_cnt = nfp_member_count; + group->to_remove = false; + + pthread_mutex_unlock(&nfp_bond->mutex); + + return 0; +} + +static void +nfp_flower_bond_group_delete(struct nfp_flower_bond *nfp_bond, + struct rte_eth_dev *bond_dev) +{ + struct nfp_bond_group *group; + + pthread_mutex_lock(&nfp_bond->mutex); + + group = nfp_flower_bond_find_group(nfp_bond, bond_dev); + if (group == NULL) { + pthread_mutex_unlock(&nfp_bond->mutex); + PMD_DRV_LOG(WARNING, "Untracked bond got unregistered %s", + bond_dev->device->name); + return; + } + + group->to_remove = true; + group->to_destroy = true; + + pthread_mutex_unlock(&nfp_bond->mutex); +} + int nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, struct rte_eth_dev *bond_dev, struct rte_eth_dev *nfp_dev, enum nfp_flower_bond_event event) { + int ret = 0; + switch (event) { + case NFP_FLOWER_CHANGEUPPER: + ret = nfp_flower_bond_changeupper_event(nfp_bond, bond_dev); + if (ret != 0) { + PMD_DRV_LOG(ERR, "Change upper event can not work."); + return ret; + } + break; case NFP_FLOWER_CHANGELINKSTATUS: nfp_flower_bond_change_linkstatus_event(nfp_bond, bond_dev, nfp_dev); break; + case NFP_FLOWER_UNREGISTER: + nfp_flower_bond_group_delete(nfp_bond, bond_dev); + break; default: PMD_DRV_LOG(ERR, "Invalid bond offload event."); return -1; @@ -288,7 +469,7 @@ nfp_flower_bond_event_handle(struct nfp_flower_bond *nfp_bond, nfp_flower_bond_do_work(nfp_bond); - return 0; + return ret; } enum nfp_flower_bond_batch diff --git a/drivers/net/nfp/flower/nfp_flower_bond.h b/drivers/net/nfp/flower/nfp_flower_bond.h index 80f56a6780..f73589265f 100644 --- a/drivers/net/nfp/flower/nfp_flower_bond.h +++ b/drivers/net/nfp/flower/nfp_flower_bond.h @@ -109,6 +109,8 @@ struct nfp_flower_bond { uint32_t global_inst; /** Incremented for each config packet sent */ uint32_t pkt_num; + /** ID of bond group in driver, true is used */ + bool group_id_map[NFP_FL_BOND_GROUP_MAX]; /** Pointer to the flower app */ struct nfp_app_fw_flower *app_fw_flower; }; diff --git a/drivers/net/nfp/flower/nfp_flower_representor.c b/drivers/net/nfp/flower/nfp_flower_representor.c index 2810a7a271..43106da90e 100644 --- a/drivers/net/nfp/flower/nfp_flower_representor.c +++ b/drivers/net/nfp/flower/nfp_flower_representor.c @@ -390,6 +390,25 @@ nfp_flower_repr_dev_close(struct rte_eth_dev *dev) return 0; } +static int +nfp_flower_repr_bond_notify_member(struct rte_eth_dev *dev, + struct rte_eth_dev *bond_dev) +{ + struct nfp_flower_representor *repr; + enum nfp_flower_bond_event event = NFP_FLOWER_CHANGEUPPER; + + if (!nfp_flower_bond_all_member_are_phyrepr(bond_dev)) + event = NFP_FLOWER_UNREGISTER; + + repr = dev->data->dev_private; + + if (nfp_flower_support_bond_offload(repr->app_fw_flower)) + return nfp_flower_bond_event_handle(repr->app_fw_flower->nfp_bond, + bond_dev, dev, event); + + return 0; +} + static const struct eth_dev_ops nfp_flower_pf_repr_dev_ops = { .dev_infos_get = nfp_flower_repr_dev_infos_get, @@ -437,6 +456,8 @@ static const struct eth_dev_ops nfp_flower_repr_dev_ops = { .flow_ops_get = nfp_flow_ops_get, .mtr_ops_get = nfp_net_mtr_ops_get, + + .bond_notify_member = nfp_flower_repr_bond_notify_member, }; static uint32_t