From patchwork Tue Sep 15 18:56:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Harman Kalra X-Patchwork-Id: 77783 X-Patchwork-Delegate: jerinj@marvell.com Return-Path: X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from dpdk.org (dpdk.org [92.243.14.124]) by inbox.dpdk.org (Postfix) with ESMTP id 74B3EA04C7; Tue, 15 Sep 2020 20:57:34 +0200 (CEST) Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DB5881C10C; Tue, 15 Sep 2020 20:57:27 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0b-0016f401.pphosted.com [67.231.156.173]) by dpdk.org (Postfix) with ESMTP id 2170C1C0C2; Tue, 15 Sep 2020 20:57:26 +0200 (CEST) Received: from pps.filterd (m0045851.ppops.net [127.0.0.1]) by mx0b-0016f401.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 08FIsqgs018316; Tue, 15 Sep 2020 11:57:25 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-type : mime-version; s=pfpt0220; bh=Y3hYMYSMyUVcoaRo1r83LkHgoCPYrynXQ/Rky0O/AxQ=; b=U4hyH9Crvw4gqOFf+UNXds3vcf6E65NMN3TYtd/1hEzFFiCsdvzf2gVvv8pN3Hf67kLv w0thNJxJxbR/DMUkHPDe1QI59BEL43kLtUntVOrBQyEEnKqOGyFBn2Hv6mAiCpU/G406 GjnEFJkfrGdMquyYG+5NOFVyb2WWK1z16g/wtmQcoH9zOqmoeqLjg79EeNjeQVSwiFXc yqCTPl1aRlxyh3C8crRFXmK6VcVUym3xktGGL1PK3mgNuahKGYufbMKBlbffYfPRTYXz c0A1lf3PRaStupzHTJ9r+4tX34u3ZA4cFIh5fKl9Y7AbHWdlE6fsiRRsxHGSCBGKtCe8 Xw== Received: from sc-exch01.marvell.com ([199.233.58.181]) by mx0b-0016f401.pphosted.com with ESMTP id 33gwun25yt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 15 Sep 2020 11:57:25 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 15 Sep 2020 11:57:23 -0700 Received: from SC-EXCH04.marvell.com (10.93.176.84) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 15 Sep 2020 11:57:23 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1497.2 via Frontend Transport; Tue, 15 Sep 2020 11:57:22 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FFUJE4iFn0AxWhK4pKskf9ViXC/mWMZA/wVcyvb4FXuC1NAT7E7x2dlQ6Lj9QkQWaMcsDN6Goa/uXvLi4bVlNkUgSXR9oj4f/smduIKQWVmPkIIJZ0VgOeUInNCtXN/4LGtYJoDbenwrcRji25j2VZ4+rgSrz6hgOkEIO2omRbiOIC/XNCjSoxmMWApTZoTeNitynQshUdXG6YP3kQL8RFnodIbyz5S12ZjtoBT9moUm2JWHNP/f49xctpWZUVApaUAES4UhXXhNuij0YObbFmgQe6GiCv9WX2BvsZf/yuj00n2RuCt2pNlAEEHZ9yHHIaVmy86yqmo9drvyRhQHiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y3hYMYSMyUVcoaRo1r83LkHgoCPYrynXQ/Rky0O/AxQ=; b=oWs3VXKi82TEhhEsvcUIPJ0DGcRIhEexwzrnDMc78Aspzqn6XBJSsGySqKPb0vx47U7rgOPbBD2icFFpdIsgHT9Fy8Cf2VnDkKY47tjbsBuThFendHC7Gmy/QV5LpcMTeKmAyCJVL3ENhTpW4eACswmIQJMrayfda4QoyfKrdQuw5ZGZUweuTX0MCJ2q08MirOBOWTGgryQXYOkou9/wm/t3NTOOaTEIqKdEO/NQWKr2Q8f4tP93JOuXvRLMyhQE7I+koBlK25OLwPE7CGEXJOBvEt1E9uikgHKsKauQr9wq07qEeaExgDkzB8bQNWEVVMd2qEw1xq/QVH+q1DCYvA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=marvell.com; dmarc=pass action=none header.from=marvell.com; dkim=pass header.d=marvell.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector1-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y3hYMYSMyUVcoaRo1r83LkHgoCPYrynXQ/Rky0O/AxQ=; b=bwvQQHsPC9IwoR5/nUgXxug290hJUUqw06NCFTF0iZ4up+FQGlt5gXdBRgis849pmcrViedk3eh7U1dso2SwwTk1SQUikqF8+9tGxHSO7LYKxHOkRR3gGTosK8xwRjIUSldGV/laaSpI10pVyXsYd9otMUzmuF5Cujo/lerDUVo= Received: from MWHPR18MB1070.namprd18.prod.outlook.com (2603:10b6:300:a5::11) by MW3PR18MB3643.namprd18.prod.outlook.com (2603:10b6:303:5a::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3370.16; Tue, 15 Sep 2020 18:57:21 +0000 Received: from MWHPR18MB1070.namprd18.prod.outlook.com ([fe80::4838:1132:399f:ea55]) by MWHPR18MB1070.namprd18.prod.outlook.com ([fe80::4838:1132:399f:ea55%12]) with mapi id 15.20.3370.019; Tue, 15 Sep 2020 18:57:21 +0000 From: Harman Kalra To: Jerin Jacob , Nithin Dabilpuram , Pavan Nikhilesh , "Kiran Kumar K" CC: , Harman Kalra , Date: Wed, 16 Sep 2020 00:26:45 +0530 Message-ID: <1600196207-31258-3-git-send-email-hkalra@marvell.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1600196207-31258-1-git-send-email-hkalra@marvell.com> References: <1600196207-31258-1-git-send-email-hkalra@marvell.com> X-ClientProxiedBy: BMXPR01CA0070.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:2c::34) To MWHPR18MB1070.namprd18.prod.outlook.com (2603:10b6:300:a5::11) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from hkarlara-OptiPlex-3046.marvell.com (115.113.156.2) by BMXPR01CA0070.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00:2c::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3370.16 via Frontend Transport; Tue, 15 Sep 2020 18:57:19 +0000 X-Mailer: git-send-email 2.7.4 X-Originating-IP: [115.113.156.2] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: af60e4a9-5491-4ae7-d891-08d859a92d17 X-MS-TrafficTypeDiagnostic: MW3PR18MB3643: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:517; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tx/a4VVnl82PFfB43teUX3HYALRdZSubbEQmX+3cpnHKHRdM8XKv4M8P/16VSw8W0qSf+XUREC6rc47y0pkbsgiLw+u8KHFi8aSddx7MuNtS2b3WFuGJPDG3djJPzWcuM/+VRShG38HkCS7UNiBGAwZuGZOe4DKEJHbFRAXRtvCqZb6Gw/ieW74zaqMsT9KlyhltsBI+dG59V/f6qqQ827B2GhnMdMryvaHA0a/xthXea+fOU+DFX2wMMqYfss/QNZHx/WTJjYNNr3nKRcauQf+WLwQsW7i5LyMbGc0KqIEM0/ZRqTY1Th0uH0/6Cti0k95uv2SxfoNbXuYYOfag9A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:MWHPR18MB1070.namprd18.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(136003)(39860400002)(366004)(376002)(396003)(346002)(2906002)(8676002)(83380400001)(186003)(16526019)(66946007)(66476007)(8936002)(52116002)(5660300002)(66556008)(7696005)(55236004)(6666004)(86362001)(450100002)(956004)(478600001)(6486002)(2616005)(110136005)(6636002)(316002)(36756003)(4326008)(26005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: jOsHlCnoX43+ZpC7Q1k5ozQpSxnz+Sbpr9i35Eb0GhDL1GkyBIpG+/C2Bm6FgJzT8TDv2lekmReRFx9Ryn3/slLMXuqYp4AILvWWyVh1j8Uu0UymCbRdRcTwr8VRhw6kO+IyN2rwXUO94dI9kGqUp3rhcxBB5u2o+zfv2yP8M5ajtfFDyO6PKgOjBJokh8InHoE6LVFEHSRyCaU0Tn7CDKzPLNX+yjSoGl9LxCq3LyEvg2pn3JMnT8IGAFuMpKuJuCxSbAqOq8JxBPH5K4HyYa8g2mILqkMIox/lOJvCXmq+2IjSznmOrelQiKv3B1E3mGSss7F8wclhLZOt6ImiF4NxdGzCuAwkFLXJeVqvtUZLNwLVp9CiU2yZAtlHkpKrGQYSGoekMBN8UQYKNc6skkxNInbOyyojyZMWGiDStEVFQrB/tMHC9YcMXOrmPojWZ1vl5QB1crPioQnB3DxzVJ7LtYxsEiribTSTEOh2cWd1u0e+yOYdW+uiFMDIsGtRQ1yCLfSsTWKPVfcRdKfnb92rZLnl+FlBH13bII0w+i4fzhSo5oZTzjTIE34Rw823DczxqaKKgPuu78eo/amt265bswAkf36gBT6H2w0iykjjFo4zg31QStlI8wnDrtqy0y/lkUcAyi8IYWrJdMoOFg== X-MS-Exchange-CrossTenant-Network-Message-Id: af60e4a9-5491-4ae7-d891-08d859a92d17 X-MS-Exchange-CrossTenant-AuthSource: MWHPR18MB1070.namprd18.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2020 18:57:21.6716 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hbvRrGVvuLtPpphkarhmBCgriTm54UrVMqGwCbwNKCMi2Ib3hUIs8Qg0LWSHlrgB5aIiUngmPJ9gHqfPj+EvBg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR18MB3643 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235, 18.0.687 definitions=2020-09-15_13:2020-09-15, 2020-09-15 signatures=0 Subject: [dpdk-dev] [PATCH 3/4] net/octeontx2: fix jumbo frame crash X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Issue has been observed in case of multi segments where mbuf data gets corrupted due to missing barriers. Changes made to mbuf just before LMTST by one core gets updatded when the same mbuf is in use by another core, leading to corruption. It should be ensured that all changes made to mbuf should be written before LMTST. Fixes: cbd5710db48d ("net/octeontx2: add Tx multi segment version") Cc: stable@dpdk.org Signed-off-by: Pavan Nikhilesh Signed-off-by: Nithin Dabilpuram Signed-off-by: Harman Kalra --- drivers/common/octeontx2/otx2_io_arm64.h | 12 ++++++++++ drivers/common/octeontx2/otx2_io_generic.h | 16 +++++++++++--- drivers/event/octeontx2/otx2_worker.h | 20 +++++++++++++---- drivers/mempool/octeontx2/otx2_mempool_ops.c | 4 ++++ drivers/net/octeontx2/otx2_tx.c | 23 ++++++++++++++------ drivers/net/octeontx2/otx2_tx.h | 23 ++++++++++++++++++++ 6 files changed, 84 insertions(+), 14 deletions(-) diff --git a/drivers/common/octeontx2/otx2_io_arm64.h b/drivers/common/octeontx2/otx2_io_arm64.h index 7e45329b3..b5c85d9a6 100644 --- a/drivers/common/octeontx2/otx2_io_arm64.h +++ b/drivers/common/octeontx2/otx2_io_arm64.h @@ -63,6 +63,18 @@ otx2_lmt_submit(rte_iova_t io_address) return result; } +static __rte_always_inline uint64_t +otx2_lmt_submit_release(rte_iova_t io_address) +{ + uint64_t result; + + asm volatile ( + ".cpu generic+lse\n" + "ldeorl xzr,%x[rf],[%[rs]]" : + [rf] "=r"(result) : [rs] "r"(io_address)); + return result; +} + static __rte_always_inline void otx2_lmt_mov(void *out, const void *in, const uint32_t lmtext) { diff --git a/drivers/common/octeontx2/otx2_io_generic.h b/drivers/common/octeontx2/otx2_io_generic.h index b1d754008..da64c9b31 100644 --- a/drivers/common/octeontx2/otx2_io_generic.h +++ b/drivers/common/octeontx2/otx2_io_generic.h @@ -45,12 +45,22 @@ otx2_lmt_submit(uint64_t io_address) return 0; } +static inline int64_t +otx2_lmt_submit_release(uint64_t io_address) +{ + RTE_SET_USED(io_address); + + return 0; +} + static __rte_always_inline void otx2_lmt_mov(void *out, const void *in, const uint32_t lmtext) { - RTE_SET_USED(out); - RTE_SET_USED(in); - RTE_SET_USED(lmtext); + /* Copy four words if lmtext = 0 + * six words if lmtext = 1 + * eight words if lmtext =2 + */ + memcpy(out, in, (4 + (2 * lmtext)) * sizeof(uint64_t)); } static __rte_always_inline void diff --git a/drivers/event/octeontx2/otx2_worker.h b/drivers/event/octeontx2/otx2_worker.h index 32d611458..41e80182d 100644 --- a/drivers/event/octeontx2/otx2_worker.h +++ b/drivers/event/octeontx2/otx2_worker.h @@ -280,7 +280,19 @@ otx2_ssogws_event_tx(struct otx2_ssogws *ws, struct rte_event ev[], /* Perform header writes before barrier for TSO */ otx2_nix_xmit_prepare_tso(m, flags); - rte_cio_wmb(); + /* Lets commit any changes in the packet here in case of single seg as + * no further changes to mbuf will be done. + * While for multi seg all mbufs used are set to NULL in + * otx2_nix_prepare_mseg() after preparing the sg list and these changes + * should be committed before LMTST. + * Also in no fast free case some mbuf fields are updated in + * otx2_nix_prefree_seg + * Hence otx2_nix_xmit_submit_lmt_release/otx2_nix_xmit_mseg_one_release + * has store barrier for multiseg. + */ + if (!(flags & NIX_TX_MULTI_SEG_F) && + !(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_cio_wmb(); txq = otx2_ssogws_xtract_meta(m, txq_data); otx2_ssogws_prepare_pkt(txq, m, cmd, flags); @@ -291,12 +303,12 @@ otx2_ssogws_event_tx(struct otx2_ssogws *ws, struct rte_event ev[], if (!ev->sched_type) { otx2_nix_xmit_mseg_prep_lmt(cmd, txq->lmt_addr, segdw); otx2_ssogws_head_wait(ws); - if (otx2_nix_xmit_submit_lmt(txq->io_addr) == 0) + if (otx2_nix_xmit_submit_lmt_release(txq->io_addr) == 0) otx2_nix_xmit_mseg_one(cmd, txq->lmt_addr, txq->io_addr, segdw); } else { - otx2_nix_xmit_mseg_one(cmd, txq->lmt_addr, txq->io_addr, - segdw); + otx2_nix_xmit_mseg_one_release(cmd, txq->lmt_addr, + txq->io_addr, segdw); } } else { /* Passing no of segdw as 4: HDR + EXT + SG + SMEM */ diff --git a/drivers/mempool/octeontx2/otx2_mempool_ops.c b/drivers/mempool/octeontx2/otx2_mempool_ops.c index 5229a7cfb..de67aa3bd 100644 --- a/drivers/mempool/octeontx2/otx2_mempool_ops.c +++ b/drivers/mempool/octeontx2/otx2_mempool_ops.c @@ -15,6 +15,10 @@ otx2_npa_enq(struct rte_mempool *mp, void * const *obj_table, unsigned int n) const uint64_t addr = npa_lf_aura_handle_to_base(aura_handle) + NPA_LF_AURA_OP_FREE0; + /* Ensure mbuf init changes are written before the free pointers + * are enqueued to the stack. + */ + rte_cio_wmb(); for (index = 0; index < n; index++) otx2_store_pair((uint64_t)obj_table[index], reg, addr); diff --git a/drivers/net/octeontx2/otx2_tx.c b/drivers/net/octeontx2/otx2_tx.c index 1af6fa649..ee5df48d3 100644 --- a/drivers/net/octeontx2/otx2_tx.c +++ b/drivers/net/octeontx2/otx2_tx.c @@ -38,8 +38,11 @@ nix_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, otx2_nix_xmit_prepare_tso(tx_pkts[i], flags); } - /* Lets commit any changes in the packet */ - rte_cio_wmb(); + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_cio_wmb(); for (i = 0; i < pkts; i++) { otx2_nix_xmit_prepare(tx_pkts[i], cmd, flags); @@ -74,12 +77,11 @@ nix_xmit_pkts_mseg(void *tx_queue, struct rte_mbuf **tx_pkts, otx2_nix_xmit_prepare_tso(tx_pkts[i], flags); } - /* Lets commit any changes in the packet */ - rte_cio_wmb(); - for (i = 0; i < pkts; i++) { otx2_nix_xmit_prepare(tx_pkts[i], cmd, flags); segdw = otx2_nix_prepare_mseg(tx_pkts[i], cmd, flags); + /* Lets commit any changes in the packet */ + rte_cio_wmb(); otx2_nix_xmit_prepare_tstamp(cmd, &txq->cmd[0], tx_pkts[i]->ol_flags, segdw, flags); @@ -127,8 +129,11 @@ nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, /* Reduce the cached count */ txq->fc_cache_pkts -= pkts; - /* Lets commit any changes in the packet */ - rte_cio_wmb(); + /* Lets commit any changes in the packet here as no further changes + * to the packet will be done unless no fast free is enabled. + */ + if (!(flags & NIX_TX_OFFLOAD_MBUF_NOFF_F)) + rte_cio_wmb(); senddesc01_w0 = vld1q_dup_u64(&txq->cmd[0]); senddesc23_w0 = senddesc01_w0; @@ -221,6 +226,10 @@ nix_xmit_pkts_vector(void *tx_queue, struct rte_mbuf **tx_pkts, 1, 0); senddesc01_w0 = vorrq_u64(senddesc01_w0, xmask01); senddesc23_w0 = vorrq_u64(senddesc23_w0, xmask23); + /* Ensuring mbuf fields which got updated in + * otx2_nix_prefree_seg are written before LMTST. + */ + rte_cio_wmb(); } else { struct rte_mbuf *mbuf; /* Mark mempool object as "put" since diff --git a/drivers/net/octeontx2/otx2_tx.h b/drivers/net/octeontx2/otx2_tx.h index caf170fd1..5b78d2e2d 100644 --- a/drivers/net/octeontx2/otx2_tx.h +++ b/drivers/net/octeontx2/otx2_tx.h @@ -363,6 +363,10 @@ otx2_nix_xmit_prepare(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) * DF bit = 0 otherwise */ send_hdr->w0.df = otx2_nix_prefree_seg(m); + /* Ensuring mbuf fields which got updated in + * otx2_nix_prefree_seg are written before LMTST. + */ + rte_cio_wmb(); } /* Mark mempool object as "put" since it is freed by NIX */ if (!send_hdr->w0.df) @@ -395,6 +399,12 @@ otx2_nix_xmit_submit_lmt(const rte_iova_t io_addr) return otx2_lmt_submit(io_addr); } +static __rte_always_inline uint64_t +otx2_nix_xmit_submit_lmt_release(const rte_iova_t io_addr) +{ + return otx2_lmt_submit_release(io_addr); +} + static __rte_always_inline uint16_t otx2_nix_prepare_mseg(struct rte_mbuf *m, uint64_t *cmd, const uint16_t flags) { @@ -483,6 +493,19 @@ otx2_nix_xmit_mseg_one(uint64_t *cmd, void *lmt_addr, } while (lmt_status == 0); } +static __rte_always_inline void +otx2_nix_xmit_mseg_one_release(uint64_t *cmd, void *lmt_addr, + rte_iova_t io_addr, uint16_t segdw) +{ + uint64_t lmt_status; + + rte_cio_wmb(); + do { + otx2_lmt_mov_seg(lmt_addr, (const void *)cmd, segdw); + lmt_status = otx2_lmt_submit(io_addr); + } while (lmt_status == 0); +} + #define L3L4CSUM_F NIX_TX_OFFLOAD_L3_L4_CSUM_F #define OL3OL4CSUM_F NIX_TX_OFFLOAD_OL3_OL4_CSUM_F #define VLAN_F NIX_TX_OFFLOAD_VLAN_QINQ_F