From patchwork Thu Sep 13 06:08:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gagandeep Singh X-Patchwork-Id: 44643 X-Patchwork-Delegate: gakhil@marvell.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DB8E25F0D; Thu, 13 Sep 2018 08:09:45 +0200 (CEST) Received: from EUR02-VE1-obe.outbound.protection.outlook.com (mail-eopbgr20055.outbound.protection.outlook.com [40.107.2.55]) by dpdk.org (Postfix) with ESMTP id 9C2DF58CB for ; Thu, 13 Sep 2018 08:09:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=k7Om6u8bqruK5cnR0DmuPlVVgK61JauyX5pWDXl/HMk=; b=V1UV1tTy33JJPpXiPm3GiuGNfqL1S1bJM7q/Et3+9K/YRIWFYCdr+C2rKxTSG8+9YPly8UtFyrjWvAcMVXWkmVfinNOEA6fY+p80ktfb+lvq1m7JuU17tixzA0gUma/NwW6wnAXZP4jgtZMAzLevPdS7ApePskKJiRtHlcXKuK0= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=G.Singh@nxp.com; Received: from Tophie.ap.freescale.net (14.142.187.166) by HE1PR04MB1529.eurprd04.prod.outlook.com (2a01:111:e400:59a8::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1143.15; Thu, 13 Sep 2018 06:09:33 +0000 From: Gagandeep Singh To: dev@dpdk.org, akhil.goyal@nxp.com Cc: Hemant Agrawal , Gagandeep Singh Date: Thu, 13 Sep 2018 11:38:43 +0530 Message-Id: <20180913060846.29930-8-g.singh@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180913060846.29930-1-g.singh@nxp.com> References: <20180913060846.29930-1-g.singh@nxp.com> MIME-Version: 1.0 X-Originating-IP: [14.142.187.166] X-ClientProxiedBy: BM1PR01CA0095.INDPRD01.PROD.OUTLOOK.COM (2603:1096:b00::11) To HE1PR04MB1529.eurprd04.prod.outlook.com (2a01:111:e400:59a8::19) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 545f8fd4-ee1f-4cc8-291a-08d6193f7a23 X-MS-Office365-Filtering-HT: Tenant X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989137)(4534165)(7168020)(4627221)(201703031133081)(201702281549075)(8990107)(5600074)(711020)(4618075)(2017052603328)(7153060)(7193020); SRVR:HE1PR04MB1529; X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1529; 3:/eFp3CqbUAHgaI/krfcQXruhv41jd0EFw8gelHBpRsoCM+5Pkr3UsmFgnClF7I7DBrlGECh+mUPWVG/hf/kcoZwo7+f0ewerMm652SW17+L+WapLtEHxC5/p+Q/x1d59nSgutj/pxplRQJBEDFLZER+pWll0Vyvb3JBC3EtelwC/Qt5DBF5UpejeM5KTKlh6B4+nU7w8Tk2FaMQiYrdjo+OCNeY5iJorVMpf39oGJvABKuD4RIyYUX8LuyWBBmeM; 25:CndY17vwIoIbYq/3UdFjpwA5BZ7MPV6D8WKog00oiocz0zStXKd3Yi509G0BzDeajaICQfEkzu7SlrsWDjnDnZtWHMrYeS6Md8s6gotDWNrgRS0757D0JKc9l3SSfQyl2IOpBlUbIDm70TK1AWNsPpbWHNLfyhavf9NQvQe2lmRvUN0h3XKHFmLgP+98gwey4yjrvbKCxrvLXmnshpFMiTut9Fg/YwOywM9LYmQYdpqFK3f6A88TrwUs4NtNwnmGplvVGCsdOpq/jJ+vs0VN4eC3/MACO6o9+WTf7VSXQ4zvvcoXJF0B55Z2BgpK++GiVNntuKNC0Rlj0cUvd+b4Qg==; 31:ScFSKbVz4k3Dx3zMtxWjrvvsN9vP0TJL1SN6ZWRCj7TL8RrSmVDI3FMKGoelQLqlxa3Dk951aD2cE5D93878WzAAdz4iTGVwoFRIIQZb22LS4JP7di4vIfa0o6ubvQEt3QoO0PNTnUG3LFUkRbM2YQ9SuAOJZySsvnuei1N61CgPEqaefJjbcKpsT84MP/j8Q/coVSA5TxamrsQcBiTETjFuOwlpZukXZTBk6kcVgCE= X-MS-TrafficTypeDiagnostic: HE1PR04MB1529: X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1529; 20:+ADgYIoIEmf7gvqJEwk4CzjXHHz1QgS+T9yqHJ2j8uSjlJT/7w2AxkxO6oHOcln1P00VBae7BXjdBG1RVzJV/aAGtkgdKsjRNe/RBqqtXfg9GRaWT71LyNt8qCK+UfWIsipZ5K/o2dHlrCDv2xd46oUsTyl/uR8MUI8vV/XrPMQmse5NnXBY6v7Cm58GnsnNPr1h0suzHE4/cFMWamCp5iafVQA+S+DtbxILfVcSNe9od52ArHzt5yLVV5sZsgpO3tHzl/jbuSRpw0r7RCiNsN44KottTzKVB8ln/t+jX107iY+llzKJNbkLvhEgDYMZWOreJ9kLXMhQHu59Vh6RVLPR6M4IcnB9ywLCgItLjCWiee3J3RuCYksT6+vIHhJ2aSArO9wd+ZJtOKlPoUe9hbcf536a0N3W35uvygy4qpn2Yeh5cKh8KhLR/IYHbAo0/6KJi1mIVAglhmkokIqsVjFwGSo60o/Kat6GjkbpZtRr8fwObiIxRovOztnr8j6E; 4:WHUMf6GTjfaIBOBC8nj9aVCklXUMWIHNBcvMIN0G1FicZMZm0gGePXJDBhmHhuZe21//P92KQmEbk93pam9Amhfoyeer8TBC2Ru0kVR68OkIbKAtzZWZb0UlhW67Ci8NCq6POCllif/ibqaVmUZa8f0U85NODDQ6meAH3WKOdXOPpatuU9OUgJz8iXHg3V3OulRdIHdztRj92kOMxg05nd1Njg6VSGcEC/Y+hBDfUG4RgC8Dr/RDYU4XVfbAzDHZWRlnpasGYp7q4Epcz2CBfxetx+G9BICrKnZvA9kdU/+3sWCmbZAX9nsC8IUSL4er X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(185117386973197); X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(5005006)(8121501046)(823301075)(3231311)(944501410)(52105095)(3002001)(93006095)(93001095)(10201501046)(6055026)(149027)(150027)(6041310)(20161123558120)(20161123562045)(20161123560045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(201708071742011)(7699050); SRVR:HE1PR04MB1529; BCL:0; PCL:0; RULEID:; SRVR:HE1PR04MB1529; X-Forefront-PRVS: 07943272E1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(366004)(376002)(39860400002)(136003)(346002)(396003)(189003)(199004)(2906002)(6506007)(386003)(5009440100003)(6486002)(51416003)(6512007)(52116002)(486006)(2616005)(68736007)(956004)(53936002)(476003)(446003)(11346002)(305945005)(26005)(7736002)(55236004)(76176011)(186003)(16526019)(50226002)(478600001)(47776003)(1076002)(3846002)(6116002)(48376002)(105586002)(106356001)(25786009)(14444005)(50466002)(5660300001)(36756003)(72206003)(66066001)(316002)(97736004)(8936002)(81166006)(4326008)(8676002)(86362001)(6636002)(6666003)(575784001)(54906003)(81156014)(16586007)(110426005)(134034003); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR04MB1529; H:Tophie.ap.freescale.net; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; Received-SPF: None (protection.outlook.com: nxp.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; HE1PR04MB1529; 23:B6oMbLuTklqNwhz6le97Zp1+sscPIHMunQ4ttblVy?= IYn+98OF4ypdma2xOmCnle7FCmhFE0uCRqAUsQknhsHcZNVw2+l0r5srNCF/lAXAklCSzcRXgn4MxkT2Z7ZUDuJOci3dYJkH3mSaht7pbEA/DYIfPckeHrPhfjPkBHuh35luLn1WrXghybOO2jMZUMswmBBhCZPOi/ANfxum/mXVjeaedry1WA6/GRuSIGLkAIVSA+IbWD+qNs5mr+YMhQuKycFRRC8i6B8mtsfo8tW3hc/rqzfNDDAAEwqERKObRk/Y9+oM3gpZhIaYyZEN6kYWNC9qBPNb3ouJFkQuTTpAalfQfIxuz0lO8Z0Y+5a/XdO2FB85mJ1uXJy73E2HAd4Q0x9aJuDSqBCOhYGLthT3Wz+GM97N1mMPuRXMiGyMxc6IFgin2FtmaqvbiIzvOobV2UalsQoI3KVCPnyWA0WSdIhD4tLhlNUJZHr7iBO+mKjpYSRqGrrNqLEy3bQt81m9kMDB9H+aZBLai+0IG7nAwOoN2vwyqoj8gTbTZXno5cT8pH9gfRms6tY7hT9NeisaSiHLuH4g2lqgXEArlqO6qBRCtOEqkmwx82lAYtZRdXp8bPDPDx/MUBsmV3RcV84MtJDTzcvsyMGsR6n+uije8jd/Utqu9WCIZRX/VJUaiq792TnDdA2i5r0dk9ZB2r4+EZnr+920uri0bk28DN5cpPc0pohs2xTmAh3RTp0KraIz1t9orljPKTv3cc8rMuFkKDnZmYicujfgj4KGoPZaWS1EEH0sz+jxEEVGXKbSrOyCuAsKiZXk/Pko4QcBc0u6PYXgqSC/68bK3HMVhEfgckoIhF02s+3HdZAa1N6qyL9OytInXVPX3gMfOsjg5uLx4pqjsF1y/ndgbtVuP6EFH/h8p/6TxlCHFvFI+ziYZj0NdecJp2eR0U5xB5Q/m8l89fhyEdKJ4XNCSFhXJhd1qRRWlc89t9FRWSuFjulhsNSQ5//RgE9rSulAC2ubj+wcodJ7KOuvN/QhADu7AyL+Ayd2lrQE+Zh659FXymUaAs2MG+PD20KFL3kf+WWaZDPdEGJukBMrWi1eChVZBfWEnw4/lnidjKBV88KCH+OQJ4IP/SD6UUj1/sLeLboqAlYlrirLbIeAVk6Jt3zSp3x8Ra0j1rCxn1dFcQsnpyqRKsSbe0Fm55dy0ylh+Mgb4eXWtNuxR3JuMfkvx3/VbvQP0H4j6CUCv80xaMKOOl/GHpnkP6R/7Uz9C8oQHRNOUENS7NMd1ohmC3OMhNhyRr1WU86IT4+xiGVoy+FrluAKpvdZG/4HLhQB5809mvBW2a1TUHpXKt0mscajkLIxiy4hJELKFflsxF7WAeDDpvunM8= X-Microsoft-Antispam-Message-Info: rUot0h5poEb5+CGHdUwj5g9EdwuMH9WOaWKxInB62flKaK3BI+Y/s4sf6HRrbgwXVCMDDa6XcnUYcxiBhaTkKi9Ulk1vQ+Wj4aNeUPPjmLN0nqkHNfcAGRp/4ow/A71liHTt+WnXQVQg5wsU2cr7CuiQYqMDEzzTG7rfDaLOALZ4mbD7amTUewyO8akKB4mgaDFlz/6X/pcLg1rl3MEtPcuEJojNZ03eah0K0o44c8SXtWnMmyBU1ghfY49Y+tgxY80NHRhG2smZMXkZ08Q5WP4yHTpFZD9IyRADP4w6r77zURnElFvuRIHu17gcQdTJW37qOGrGjGLduuMNK8DKwoqTj0kgrs6//XP6D63WSYo= X-Microsoft-Exchange-Diagnostics: 1; HE1PR04MB1529; 6:P8t2IL9RkWveOgCi/rQuW4DLu/OjR8itBaQq47l+vy4pUxgHiADguenp015Kgxtb5bz9e72X50f4DHIGQAdKzncOEaOfwBFx3mfTOp6u0BXc8LpI46XrK0aZFDSVlsvhRmibXLo2kU4hgspw7yKaHhg5kSB/X8W5ehSZJ9YOUO+Wt7o4qfxmtUTMWY1X+WNUhA4mqAx6nRjWsjDsqj9dz07aiRsbQT/Ask2t5I/57ow6ylEsTyaNRuGrsUG16xcN0+LLJFB4ds+FU6EB+l4gbvv6p708T7v/5Wv/OjiraDLEzv6IVjrxGxwLJSKHqDVV5DKGHdsjTAMy+R3AmMePIQ1Abv61WfXzlCkazTkPHQ0F7LVqw9WXQ3pTgqlR2kA9ykrTzehpv+YiYzncFkNirGigLayOilVSCSg+ex8Heygzk7FouIKby/v4kTzQLy3/cRlYd+N1mUTmYtUIjLhxNA==; 5:p+0o+/DyWxM8cVBpcbLM0IIJx1h0VDVJC4pUD7jDuWUuIDoCHkgps8ochiltUFoToFmcKhTd7kIG2ykQnv94bdXhnnwEBt5Dv2c0PumfD0taU9tTfJypc0DiOXaEZXUDEwiZjktA9b+GaCj1bV1AVfTWGUgp514mCsqfOeepvWY=; 7:DahASQgbH/vfhXTuB0uYdnfdNcOmBveViWD9FZTUmp8bdQwGDRpwDwx5TIkmJKsf3Ef2TAo72OwuGmBPqFHgV9kdNbKxKLupfOG5bh6Scq63ObfcpW13G9kkdrrGoVdoyXCNQY86nf7UVGJPcPKavL61GJNPY3UahvQiQHwIxXo/QcfsbhAjFkzXcf6K0In2b7kGdbpZriypCciiFExB9kS8IilmWdcwljokFvxvPa1kqWRLaoyoz2z2qK384d3h SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Sep 2018 06:09:33.0543 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 545f8fd4-ee1f-4cc8-291a-08d6193f7a23 X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR04MB1529 Subject: [dpdk-dev] [PATCH 07/10] crypto/caam_jr: add enqueue and dequeue routines X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" From: Hemant Agrawal Signed-off-by: Gagandeep Singh Signed-off-by: Hemant Agrawal --- drivers/crypto/caam_jr/caam_jr.c | 621 +++++++++++++++++++++++++- drivers/crypto/caam_jr/caam_jr_desc.h | 289 ++++++++++++ 2 files changed, 908 insertions(+), 2 deletions(-) create mode 100644 drivers/crypto/caam_jr/caam_jr_desc.h diff --git a/drivers/crypto/caam_jr/caam_jr.c b/drivers/crypto/caam_jr/caam_jr.c index a0eee3b85..6d30c4f4d 100644 --- a/drivers/crypto/caam_jr/caam_jr.c +++ b/drivers/crypto/caam_jr/caam_jr.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #define CRYPTODEV_NAME_CAAM_JR_PMD crypto_caam_jr @@ -50,6 +51,142 @@ static enum sec_driver_state_e g_driver_state = SEC_DRIVER_STATE_IDLE; static int g_job_rings_no; static int g_job_rings_max; +struct sec_outring_entry { + phys_addr_t desc; /* Pointer to completed descriptor */ + uint32_t status; /* Status for completed descriptor */ +} __rte_packed; + +/* virtual address conversin when mempool support is available for ctx */ +static inline phys_addr_t +caam_jr_vtop_ctx(struct caam_jr_op_ctx *ctx, void *vaddr) +{ + return (size_t)vaddr - ctx->vtop_offset; +} + +static inline void +caam_jr_op_ending(struct caam_jr_op_ctx *ctx) +{ + /* report op status to sym->op and then free the ctx memeory */ + rte_mempool_put(ctx->ctx_pool, (void *)ctx); +} + +static inline struct caam_jr_op_ctx * +caam_jr_alloc_ctx(struct caam_jr_session *ses) +{ + struct caam_jr_op_ctx *ctx; + int ret; + + ret = rte_mempool_get(ses->ctx_pool, (void **)(&ctx)); + if (!ctx || ret) { + CAAM_JR_DP_WARN("Alloc sec descriptor failed!"); + return NULL; + } + /* + * Clear SG memory. There are 16 SG entries of 16 Bytes each. + * one call to dcbz_64() clear 64 bytes, hence calling it 4 times + * to clear all the SG entries. caam_jr_alloc_ctx() is called for + * each packet, memset is costlier than dcbz_64(). + */ + dcbz_64(&ctx->sg[SG_CACHELINE_0]); + dcbz_64(&ctx->sg[SG_CACHELINE_1]); + dcbz_64(&ctx->sg[SG_CACHELINE_2]); + dcbz_64(&ctx->sg[SG_CACHELINE_3]); + + ctx->ctx_pool = ses->ctx_pool; + ctx->vtop_offset = (size_t) ctx - rte_mempool_virt2iova(ctx); + + return ctx; +} +static inline int is_cipher_only(struct caam_jr_session *ses) +{ + return ((ses->cipher_alg != RTE_CRYPTO_CIPHER_NULL) && + (ses->auth_alg == RTE_CRYPTO_AUTH_NULL)); +} + +static inline void +caam_cipher_alg(struct caam_jr_session *ses, struct alginfo *alginfo_c) +{ + switch (ses->cipher_alg) { + case RTE_CRYPTO_CIPHER_NULL: + break; + case RTE_CRYPTO_CIPHER_AES_CBC: + alginfo_c->algtype = OP_ALG_ALGSEL_AES; + alginfo_c->algmode = OP_ALG_AAI_CBC; + break; + case RTE_CRYPTO_CIPHER_3DES_CBC: + alginfo_c->algtype = OP_ALG_ALGSEL_3DES; + alginfo_c->algmode = OP_ALG_AAI_CBC; + break; + case RTE_CRYPTO_CIPHER_AES_CTR: + alginfo_c->algtype = OP_ALG_ALGSEL_AES; + alginfo_c->algmode = OP_ALG_AAI_CTR; + break; + default: + CAAM_JR_DEBUG("unsupported cipher alg %d", ses->cipher_alg); + } +} + +/* prepare command block of the session */ +static int +caam_jr_prep_cdb(struct caam_jr_session *ses) +{ + struct alginfo alginfo_c = {0}; + int32_t shared_desc_len = 0; + struct sec_cdb *cdb; +#if RTE_BYTE_ORDER == RTE_BIG_ENDIAN + int swap = false; +#else + int swap = true; +#endif + + if (ses->cdb) + caam_jr_dma_free(ses->cdb); + + cdb = caam_jr_dma_mem_alloc(L1_CACHE_BYTES, sizeof(struct sec_cdb)); + if (!cdb) { + CAAM_JR_ERR("failed to allocate memory for cdb\n"); + return -1; + } + + ses->cdb = cdb; + + memset(cdb, 0, sizeof(struct sec_cdb)); + + if (is_cipher_only(ses)) { + caam_cipher_alg(ses, &alginfo_c); + if (alginfo_c.algtype == (unsigned int)CAAM_JR_ALG_UNSUPPORT) { + CAAM_JR_ERR("not supported cipher alg"); + rte_free(cdb); + return -ENOTSUP; + } + + alginfo_c.key = (size_t)ses->cipher_key.data; + alginfo_c.keylen = ses->cipher_key.length; + alginfo_c.key_enc_flags = 0; + alginfo_c.key_type = RTA_DATA_IMM; + + shared_desc_len = cnstr_shdsc_blkcipher( + cdb->sh_desc, true, + swap, &alginfo_c, + NULL, + ses->iv.length, + ses->dir); + } + + if (shared_desc_len < 0) { + CAAM_JR_ERR("error in preparing command block"); + return shared_desc_len; + } + +#ifdef RTE_LIBRTE_PMD_CAAM_JR_DEBUG + SEC_DUMP_DESC(cdb->sh_desc); +#endif + + cdb->sh_hdr.hi.field.idlen = shared_desc_len; + + return 0; +} + /* @brief Poll the HW for already processed jobs in the JR * and silently discard the available jobs or notify them to UA * with indicated error code. @@ -105,6 +242,486 @@ static void hw_flush_job_ring(struct sec_job_ring_t *job_ring, } } +/* @brief Poll the HW for already processed jobs in the JR + * and notify the available jobs to UA. + * + * @param [in] job_ring The job ring to poll. + * @param [in] limit The maximum number of jobs to notify. + * If set to negative value, all available jobs are + * notified. + * + * @retval >=0 for No of jobs notified to UA. + * @retval -1 for error + */ +static int +hw_poll_job_ring(struct sec_job_ring_t *job_ring, + struct rte_crypto_op **ops, int32_t limit, + struct caam_jr_qp *jr_qp) +{ + int32_t jobs_no_to_notify = 0; /* the number of done jobs to notify*/ + int32_t number_of_jobs_available = 0; + int32_t notified_descs_no = 0; + uint32_t sec_error_code = 0; + struct job_descriptor *current_desc; + phys_addr_t current_desc_addr; + phys_addr_t *temp_addr; + struct caam_jr_op_ctx *ctx; + + /* TODO check for ops have memory*/ + /* check here if any JR error that cannot be written + * in the output status word has occurred + */ + if (JR_REG_JRINT_JRE_EXTRACT(GET_JR_REG(JRINT, job_ring))) { + CAAM_JR_INFO("err received"); + sec_error_code = JR_REG_JRINT_ERR_TYPE_EXTRACT( + GET_JR_REG(JRINT, job_ring)); + if (unlikely(sec_error_code)) { + hw_job_ring_error_print(job_ring, sec_error_code); + return -1; + } + } + /* compute the number of jobs available in the job ring based on the + * producer and consumer index values. + */ + number_of_jobs_available = hw_get_no_finished_jobs(job_ring); + /* Compute the number of notifications that need to be raised to UA + * If limit > total number of done jobs -> notify all done jobs + * If limit = 0 -> error + * If limit < total number of done jobs -> notify a number + * of done jobs equal with limit + */ + jobs_no_to_notify = (limit > number_of_jobs_available) ? + number_of_jobs_available : limit; + CAAM_JR_DP_DEBUG( + "Jr[%p] pi[%d] ci[%d].limit =%d Available=%d.Jobs to notify=%d", + job_ring, job_ring->pidx, job_ring->cidx, + limit, number_of_jobs_available, jobs_no_to_notify); + + rte_smp_rmb(); + + while (jobs_no_to_notify > notified_descs_no) { + static uint64_t false_alarm; + static uint64_t real_poll; + + /* Get job status here */ + sec_error_code = job_ring->output_ring[job_ring->cidx].status; + /* Get completed descriptor */ + temp_addr = &(job_ring->output_ring[job_ring->cidx].desc); + current_desc_addr = (phys_addr_t)sec_read_addr(temp_addr); + + real_poll++; + /* todo check if it is false alarm no desc present */ + if (!current_desc_addr) { + false_alarm++; + printf("false alarm %" PRIu64 "real %" PRIu64 + " sec_err =0x%x cidx Index =0%d\n", + false_alarm, real_poll, + sec_error_code, job_ring->cidx); + rte_panic("CAAM JR descriptor NULL"); + return notified_descs_no; + } + current_desc = (struct job_descriptor *) + caam_jr_dma_ptov(current_desc_addr); + /* now increment the consumer index for the current job ring, + * AFTER saving job in temporary location! + */ + job_ring->cidx = SEC_CIRCULAR_COUNTER(job_ring->cidx, + SEC_JOB_RING_SIZE); + /* Signal that the job has been processed and the slot is free*/ + hw_remove_entries(job_ring, 1); + /*TODO for multiple ops, packets*/ + ctx = container_of(current_desc, struct caam_jr_op_ctx, jobdes); + if (unlikely(sec_error_code)) { + CAAM_JR_ERR("desc at cidx %d generated error 0x%x\n", + job_ring->cidx, sec_error_code); + hw_handle_job_ring_error(job_ring, sec_error_code); + //todo improve with exact errors + ctx->op->status = RTE_CRYPTO_OP_STATUS_ERROR; + jr_qp->rx_errs++; + } else { + ctx->op->status = RTE_CRYPTO_OP_STATUS_SUCCESS; +#ifdef RTE_LIBRTE_PMD_CAAM_JR_DEBUG + if (ctx->op->sym->m_dst) { + rte_hexdump(stdout, "PROCESSED", + rte_pktmbuf_mtod(ctx->op->sym->m_dst, void *), + rte_pktmbuf_data_len(ctx->op->sym->m_dst)); + } else { + rte_hexdump(stdout, "PROCESSED", + rte_pktmbuf_mtod(ctx->op->sym->m_src, void *), + rte_pktmbuf_data_len(ctx->op->sym->m_src)); + } +#endif + } + if (ctx->op->sess_type == RTE_CRYPTO_OP_SECURITY_SESSION) { + struct ip *ip4_hdr; + + if (ctx->op->sym->m_dst) { + /*TODO check for ip header or other*/ + ip4_hdr = (struct ip *)rte_pktmbuf_mtod(ctx->op->sym->m_dst, char*); + ctx->op->sym->m_dst->pkt_len = + rte_be_to_cpu_16(ip4_hdr->ip_len); + ctx->op->sym->m_dst->data_len = + rte_be_to_cpu_16(ip4_hdr->ip_len); + } else { + ip4_hdr = (struct ip *)rte_pktmbuf_mtod(ctx->op->sym->m_src, char*); + ctx->op->sym->m_src->pkt_len = + rte_be_to_cpu_16(ip4_hdr->ip_len); + ctx->op->sym->m_src->data_len = + rte_be_to_cpu_16(ip4_hdr->ip_len); + } + } + *ops = ctx->op; + caam_jr_op_ending(ctx); + ops++; + notified_descs_no++; + } + return notified_descs_no; +} + +static uint16_t +caam_jr_dequeue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + struct caam_jr_qp *jr_qp = (struct caam_jr_qp *)qp; + struct sec_job_ring_t *ring = jr_qp->ring; + int num_rx; + int ret; + + CAAM_JR_DP_DEBUG("Jr[%p]Polling. limit[%d]", ring, nb_ops); + + /* Poll job ring + * If nb_ops < 0 -> poll JR until no more notifications are available. + * If nb_ops > 0 -> poll JR until limit is reached. + */ + + /* Run hw poll job ring */ + num_rx = hw_poll_job_ring(ring, ops, nb_ops, jr_qp); + if (num_rx < 0) { + CAAM_JR_ERR("Error polling SEC engine (%d)", num_rx); + return 0; + } + + CAAM_JR_DP_DEBUG("Jr[%p].Jobs notified[%d]. ", ring, num_rx); + + if (ring->jr_mode == SEC_NOTIFICATION_TYPE_NAPI) { + if (num_rx < nb_ops) { + ret = caam_jr_enable_irqs(ring->irq_fd); + SEC_ASSERT(ret == 0, ret, + "Failed to enable irqs for job ring %p", ring); + } + } else if (ring->jr_mode == SEC_NOTIFICATION_TYPE_IRQ) { + + /* Always enable IRQ generation when in pure IRQ mode */ + ret = caam_jr_enable_irqs(ring->irq_fd); + SEC_ASSERT(ret == 0, ret, + "Failed to enable irqs for job ring %p", ring); + } + + jr_qp->rx_pkts += num_rx; + + return num_rx; +} + + +static inline struct caam_jr_op_ctx * +build_cipher_only_sg(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct rte_mbuf *mbuf = sym->m_src; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg, *in_sg; + int length; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + uint8_t *IV_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + ses->iv.offset); + struct sec_job_descriptor_t *jobdescr; + uint8_t reg_segs; + + if (sym->m_dst) { + mbuf = sym->m_dst; + reg_segs = mbuf->nb_segs + sym->m_src->nb_segs + 2; + } else { + mbuf = sym->m_src; + reg_segs = mbuf->nb_segs * 2 + 2; + } + + if (reg_segs > MAX_SG_ENTRIES) { + CAAM_JR_DP_ERR("Cipher: Max sec segs supported is %d", + MAX_SG_ENTRIES); + return NULL; + } + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + +#ifdef RTE_LIBRTE_PMD_CAAM_JR_DEBUG + CAAM_JR_INFO("mbuf offset =%d, cipher offset = %d, length =%d+%d", + sym->m_src->data_off, sym->cipher.data.offset, + sym->cipher.data.length, ses->iv.length); +#endif + /* output */ + if (sym->m_dst) + mbuf = sym->m_dst; + else + mbuf = sym->m_src; + + sg = &ctx->sg[0]; + length = sym->cipher.data.length; + + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + + sym->cipher.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->cipher.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + SEC_JD_SET_OUT_PTR(jobdescr, + (uint64_t)caam_jr_vtop_ctx(ctx, &ctx->sg[0]), 0, + length); + /*enabling sg bit */ + (jobdescr)->seq_out.command.word |= 0x01000000; + + /*input */ + sg++; + mbuf = sym->m_src; + in_sg = sg; + + length = sym->cipher.data.length + ses->iv.length; + + /* IV */ + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len = cpu_to_caam32(ses->iv.length); + + /* 1st seg */ + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf) + + sym->cipher.data.offset); + sg->len = cpu_to_caam32(mbuf->data_len - sym->cipher.data.offset); + + /* Successive segs */ + mbuf = mbuf->next; + while (mbuf) { + sg++; + sg->ptr = cpu_to_caam64(rte_pktmbuf_iova(mbuf)); + sg->len = cpu_to_caam32(mbuf->data_len); + mbuf = mbuf->next; + } + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)caam_jr_vtop_ctx(ctx, in_sg), 0, + length); + /*enabling sg bit */ + (jobdescr)->seq_in.command.word |= 0x01000000; + + return ctx; +} + +static inline struct caam_jr_op_ctx * +build_cipher_only(struct rte_crypto_op *op, struct caam_jr_session *ses) +{ + struct rte_crypto_sym_op *sym = op->sym; + struct caam_jr_op_ctx *ctx; + struct sec4_sg_entry *sg; + rte_iova_t src_start_addr, dst_start_addr; + struct sec_cdb *cdb; + uint64_t sdesc_offset; + uint8_t *IV_ptr = rte_crypto_op_ctod_offset(op, uint8_t *, + ses->iv.offset); + struct sec_job_descriptor_t *jobdescr; + + ctx = caam_jr_alloc_ctx(ses); + if (!ctx) + return NULL; + + ctx->op = op; + cdb = ses->cdb; + sdesc_offset = (size_t) ((char *)&cdb->sh_desc - (char *)cdb); + + src_start_addr = rte_pktmbuf_iova(sym->m_src); + if (sym->m_dst) + dst_start_addr = rte_pktmbuf_iova(sym->m_dst); + else + dst_start_addr = src_start_addr; + + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + SEC_JD_INIT(jobdescr); + SEC_JD_SET_SD(jobdescr, + (phys_addr_t)(caam_jr_dma_vtop(cdb)) + sdesc_offset, + cdb->sh_hdr.hi.field.idlen); + +#ifdef RTE_LIBRTE_PMD_CAAM_JR_DEBUG + CAAM_JR_INFO("mbuf offset =%d, cipher offset = %d, length =%d+%d", + sym->m_src->data_off, sym->cipher.data.offset, + sym->cipher.data.length, ses->iv.length); +#endif + /* output */ + SEC_JD_SET_OUT_PTR(jobdescr, (uint64_t)dst_start_addr, + sym->cipher.data.offset, + sym->cipher.data.length + ses->iv.length); + + /*input */ + sg = &ctx->sg[0]; + SEC_JD_SET_IN_PTR(jobdescr, (uint64_t)caam_jr_vtop_ctx(ctx, sg), 0, + sym->cipher.data.length + ses->iv.length); + /*enabling sg bit */ + (jobdescr)->seq_in.command.word |= 0x01000000; + + sg->ptr = cpu_to_caam64(caam_jr_dma_vtop(IV_ptr)); + sg->len = cpu_to_caam32(ses->iv.length); + + sg = &ctx->sg[1]; + sg->ptr = cpu_to_caam64(src_start_addr + sym->cipher.data.offset); + sg->len = cpu_to_caam32(sym->cipher.data.length); + /* last element*/ + sg->len |= cpu_to_caam32(SEC4_SG_LEN_FIN); + + return ctx; +} + +static int +caam_jr_enqueue_op(struct rte_crypto_op *op, struct caam_jr_qp *qp) +{ + struct sec_job_ring_t *ring = qp->ring; + struct caam_jr_session *ses; + struct caam_jr_op_ctx *ctx = NULL; + struct sec_job_descriptor_t *jobdescr __rte_unused; + + switch (op->sess_type) { + case RTE_CRYPTO_OP_WITH_SESSION: + ses = (struct caam_jr_session *) + get_sym_session_private_data(op->sym->session, + cryptodev_driver_id); + break; + default: + CAAM_JR_DP_ERR("sessionless crypto op not supported"); + qp->tx_errs++; + return -1; + } + + if (unlikely(!ses->qp || ses->qp != qp)) { + CAAM_JR_DP_DEBUG("Old:sess->qp=%p New qp = %p\n", ses->qp, qp); + ses->qp = qp; + caam_jr_prep_cdb(ses); + } + + if (rte_pktmbuf_is_contiguous(op->sym->m_src)) { + if (is_cipher_only(ses)) + ctx = build_cipher_only(op, ses); + } else { + if (is_cipher_only(ses)) + ctx = build_cipher_only_sg(op, ses); + } + if (unlikely(!ctx)) { + qp->tx_errs++; + CAAM_JR_ERR("not supported sec op"); + return -1; + } +#ifdef RTE_LIBRTE_PMD_CAAM_JR_DEBUG + if (is_decode(ses)) + rte_hexdump(stdout, "DECODE", + rte_pktmbuf_mtod(op->sym->m_src, void *), + rte_pktmbuf_data_len(op->sym->m_src)); + else + rte_hexdump(stdout, "ENCODE", + rte_pktmbuf_mtod(op->sym->m_src, void *), + rte_pktmbuf_data_len(op->sym->m_src)); + + printf("\n JD before conversion\n"); + for (int i = 0; i < 12; i++) + printf("\n 0x%08x", ctx->jobdes.desc[i]); +#endif + + CAAM_JR_DP_DEBUG("Jr[%p] pi[%d] ci[%d].Before sending desc", + ring, ring->pidx, ring->cidx); + + /* todo - do we want to retry */ + if (SEC_JOB_RING_IS_FULL(ring->pidx, ring->cidx, + SEC_JOB_RING_SIZE, SEC_JOB_RING_SIZE)) { + CAAM_JR_DP_DEBUG("Ring FULL Jr[%p] pi[%d] ci[%d].Size = %d", + ring, ring->pidx, ring->cidx, SEC_JOB_RING_SIZE); + caam_jr_op_ending(ctx); + qp->tx_ring_full++; + return -EBUSY; + } + +#if CORE_BYTE_ORDER != CAAM_BYTE_ORDER + jobdescr = (struct sec_job_descriptor_t *) ctx->jobdes.desc; + + jobdescr->deschdr.command.word = + cpu_to_caam32(jobdescr->deschdr.command.word); + jobdescr->sd_ptr = cpu_to_caam64(jobdescr->sd_ptr); + jobdescr->seq_out.command.word = + cpu_to_caam32(jobdescr->seq_out.command.word); + jobdescr->seq_out_ptr = cpu_to_caam64(jobdescr->seq_out_ptr); + jobdescr->out_ext_length = cpu_to_caam32(jobdescr->out_ext_length); + jobdescr->seq_in.command.word = + cpu_to_caam32(jobdescr->seq_in.command.word); + jobdescr->seq_in_ptr = cpu_to_caam64(jobdescr->seq_in_ptr); + jobdescr->in_ext_length = cpu_to_caam32(jobdescr->in_ext_length); + jobdescr->load_dpovrd.command.word = + cpu_to_caam32(jobdescr->load_dpovrd.command.word); + jobdescr->dpovrd = cpu_to_caam32(jobdescr->dpovrd); +#endif + + /* Set ptr in input ring to current descriptor */ + sec_write_addr(&ring->input_ring[ring->pidx], + (phys_addr_t)caam_jr_vtop_ctx(ctx, ctx->jobdes.desc)); + rte_smp_wmb(); + + /* Notify HW that a new job is enqueued */ + hw_enqueue_desc_on_job_ring(ring); + + /* increment the producer index for the current job ring */ + ring->pidx = SEC_CIRCULAR_COUNTER(ring->pidx, SEC_JOB_RING_SIZE); + + return 0; +} + +static uint16_t +caam_jr_enqueue_burst(void *qp, struct rte_crypto_op **ops, + uint16_t nb_ops) +{ + /* Function to transmit the frames to given device and queuepair */ + uint32_t loop; + int32_t ret; + struct caam_jr_qp *jr_qp = (struct caam_jr_qp *)qp; + uint16_t num_tx = 0; + /*Prepare each packet which is to be sent*/ + for (loop = 0; loop < nb_ops; loop++) { + ret = caam_jr_enqueue_op(ops[loop], jr_qp); + if (!ret) + num_tx++; + } + + jr_qp->tx_pkts += num_tx; + + return num_tx; +} + /* Release queue pair */ static int caam_jr_queue_pair_release(struct rte_cryptodev *dev, @@ -642,8 +1259,8 @@ caam_jr_dev_init(const char *name, dev->dev_ops = &caam_jr_ops; /* register rx/tx burst functions for data path */ - dev->dequeue_burst = NULL; - dev->enqueue_burst = NULL; + dev->dequeue_burst = caam_jr_dequeue_burst; + dev->enqueue_burst = caam_jr_enqueue_burst; dev->feature_flags = RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO | RTE_CRYPTODEV_FF_HW_ACCELERATED | RTE_CRYPTODEV_FF_SYM_OPERATION_CHAINING | diff --git a/drivers/crypto/caam_jr/caam_jr_desc.h b/drivers/crypto/caam_jr/caam_jr_desc.h new file mode 100644 index 000000000..47c3adf72 --- /dev/null +++ b/drivers/crypto/caam_jr/caam_jr_desc.h @@ -0,0 +1,289 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright 2017-2018 NXP + */ + +#ifndef CAAM_JR_DESC_H +#define CAAM_JR_DESC_H + +#define CMD_HDR_CTYPE_SD 0x16 +#define CMD_HDR_CTYPE_JD 0x17 + +/* The maximum size of a SEC descriptor, in WORDs (32 bits). */ +#define MAX_DESC_SIZE_WORDS 64 + +/* + * Macros manipulating descriptors + */ +/* Macro for setting the SD pointer in a JD. Common for all protocols + * supported by the SEC driver. + */ +#define SEC_JD_SET_SD(descriptor, ptr, len) { \ + (descriptor)->sd_ptr = (ptr); \ + (descriptor)->deschdr.command.jd.shr_desc_len = (len); \ +} + +/* Macro for setting a pointer to the job which this descriptor processes. + * It eases the lookup procedure for identifying the descriptor that has + * completed. + */ +#define SEC_JD_SET_JOB_PTR(descriptor, ptr) \ + ((descriptor)->job_ptr = (ptr)) + +/* Macro for setting up a JD. The structure of the JD is common across all + * supported protocols, thus its structure is identical. + */ +#define SEC_JD_INIT(descriptor) ({ \ + /* CTYPE = job descriptor \ + * RSMS, DNR = 0 + * ONE = 1 + * Start Index = 0 + * ZRO,TD, MTD = 0 + * SHR = 1 (there's a shared descriptor referenced + * by this job descriptor,pointer in next word) + * REO = 1 (execute job descr. first, shared descriptor + * after) + * SHARE = DEFER + * Descriptor Length = 0 ( to be completed @ runtime ) + */ \ + (descriptor)->deschdr.command.word = 0xB0801C0D; \ + /* + * CTYPE = SEQ OUT command * Scater Gather Flag = 0 + * (can be updated @ runtime) PRE = 0 * EXT = 1 + * (data length is in next word, following the * command) + * RTO = 0 + */ \ + (descriptor)->seq_out.command.word = 0xF8400000; /**/ \ + /* + * CTYPE = SEQ IN command + * Scater Gather Flag = 0 (can be updated @ runtime) + * PRE = 0 + * EXT = 1 ( data length is in next word, following the + * command) + * RTO = 0 + */ \ + (descriptor)->seq_in.command.word = 0xF0400000; /**/ \ + /* + * In order to be compatible with QI scenarios, the DPOVRD value + * loaded must be formated like this: + * DPOVRD_EN (1b) | Res| DPOVRD Value (right aligned). + */ \ + (descriptor)->load_dpovrd.command.word = 0x16870004; \ + /* By default, DPOVRD mechanism is disabled, thus the value to be + * LOAD-ed through the above descriptor command will be 0x0000_0000. + */ \ + (descriptor)->dpovrd = 0x00000000; \ +}) + +/* Macro for setting the pointer to the input buffer in the JD, according to + * the parameters set by the user in the ::sec_packet_t structure. + */ +#define SEC_JD_SET_IN_PTR(descriptor, phys_addr, offset, length) { \ + (descriptor)->seq_in_ptr = (phys_addr) + (offset); \ + (descriptor)->in_ext_length = (length); \ +} + +/* Macro for setting the pointer to the output buffer in the JD, according to + * the parameters set by the user in the ::sec_packet_t structure. + */ +#define SEC_JD_SET_OUT_PTR(descriptor, phys_addr, offset, length) { \ + (descriptor)->seq_out_ptr = (phys_addr) + (offset); \ + (descriptor)->out_ext_length = (length); \ +} + +/* Macro for setting the Scatter-Gather flag in the SEQ IN command. Used in + * case the input buffer is split in multiple buffers, according to the user + * specification. + */ +#define SEC_JD_SET_SG_IN(descriptor) \ + ((descriptor)->seq_in.command.field.sgf = 1) + +/* Macro for setting the Scatter-Gather flag in the SEQ OUT command. Used in + * case the output buffer is split in multiple buffers, according to the user + * specification. + */ +#define SEC_JD_SET_SG_OUT(descriptor) \ + ((descriptor)->seq_out.command.field.sgf = 1) + +#define SEC_JD_SET_DPOVRD(descriptor) \ + +/* Macro for retrieving a descriptor's length. Works for both SD and JD. */ +#define SEC_GET_DESC_LEN(descriptor) \ + (((struct descriptor_header_s *)(descriptor))->command.sd.ctype == \ + CMD_HDR_CTYPE_SD ? ((struct descriptor_header_s *) \ + (descriptor))->command.sd.desclen : \ + ((struct descriptor_header_s *)(descriptor))->command.jd.desclen) + +/* Helper macro for dumping the hex representation of a descriptor */ +#define SEC_DUMP_DESC(descriptor) { \ + int __i; \ + CAAM_JR_INFO("Des@ 0x%08x\n", (uint32_t)((uint32_t *)(descriptor)));\ + for (__i = 0; \ + __i < SEC_GET_DESC_LEN(descriptor); \ + __i++) { \ + printf("0x%08x: 0x%08x\n", \ + (uint32_t)(((uint32_t *)(descriptor)) + __i), \ + *(((uint32_t *)(descriptor)) + __i)); \ + } \ +} +/* Union describing a descriptor header. + */ +struct descriptor_header_s { + union { + uint32_t word; + struct { + /* 4 */ unsigned int ctype:5; + /* 5 */ unsigned int res1:2; + /* 7 */ unsigned int dnr:1; + /* 8 */ unsigned int one:1; + /* 9 */ unsigned int res2:1; + /* 10 */ unsigned int start_idx:6; + /* 16 */ unsigned int res3:2; + /* 18 */ unsigned int cif:1; + /* 19 */ unsigned int sc:1; + /* 20 */ unsigned int pd:1; + /* 21 */ unsigned int res4:1; + /* 22 */ unsigned int share:2; + /* 24 */ unsigned int res5:2; + /* 26 */ unsigned int desclen:6; + } sd; + struct { + /* TODO only below struct members are corrected, + * all others also need to be reversed please verify it + */ + /* 0 */ unsigned int desclen:7; + /* 7 */ unsigned int res4:1; + /* 8 */ unsigned int share:3; + /* 11 */ unsigned int reo:1; + /* 12 */ unsigned int shr:1; + /* 13 */ unsigned int mtd:1; + /* 14 */ unsigned int td:1; + /* 15 */ unsigned int zero:1; + /* 16 */ unsigned int shr_desc_len:6; + /* 22 */ unsigned int res2:1; + /* 23 */ unsigned int one:1; + /* 24 */ unsigned int dnr:1; + /* 25 */ unsigned int rsms:1; + /* 26 */ unsigned int res1:1; + /* 27 */ unsigned int ctype:5; + } jd; + } __rte_packed command; +} __rte_packed; + +/* Union describing a KEY command in a descriptor. + */ +struct key_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int cls:2; + unsigned int sgf:1; + unsigned int imm:1; + unsigned int enc:1; + unsigned int nwb:1; + unsigned int ekt:1; + unsigned int kdest:4; + unsigned int tk:1; + unsigned int rsvd1:5; + unsigned int length:10; + } __rte_packed field; + } __rte_packed command; +} __rte_packed; + +/* Union describing a PROTOCOL command + * in a descriptor. + */ +struct protocol_operation_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int optype:3; + unsigned char protid; + unsigned short protinfo; + } __rte_packed field; + } __rte_packed command; +} __rte_packed; + +/* Union describing a SEQIN command in a + * descriptor. + */ +struct seq_in_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int res1:1; + unsigned int inl:1; + unsigned int sgf:1; + unsigned int pre:1; + unsigned int ext:1; + unsigned int rto:1; + unsigned int rjd:1; + unsigned int res2:4; + unsigned int length:16; + } field; + } __rte_packed command; +} __rte_packed; + +/* Union describing a SEQOUT command in a + * descriptor. + */ +struct seq_out_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int res1:2; + unsigned int sgf:1; + unsigned int pre:1; + unsigned int ext:1; + unsigned int rto:1; + unsigned int res2:5; + unsigned int length:16; + } field; + } __rte_packed command; +} __rte_packed; + +struct load_command_s { + union { + uint32_t word; + struct { + unsigned int ctype:5; + unsigned int class:2; + unsigned int sgf:1; + unsigned int imm:1; + unsigned int dst:7; + unsigned char offset; + unsigned char length; + } fields; + } __rte_packed command; +} __rte_packed; + +/* Structure encompassing a general shared descriptor of maximum + * size (64 WORDs). Usually, other specific shared descriptor structures + * will be type-casted to this one + * this one. + */ +struct sec_sd_t { + uint32_t rsvd[MAX_DESC_SIZE_WORDS]; +} __attribute__((packed, aligned(64))); + +/* Structure encompassing a job descriptor which processes + * a single packet from a context. The job descriptor references + * a shared descriptor from a SEC context. + */ +struct sec_job_descriptor_t { + struct descriptor_header_s deschdr; + dma_addr_t sd_ptr; + struct seq_out_command_s seq_out; + dma_addr_t seq_out_ptr; + uint32_t out_ext_length; + struct seq_in_command_s seq_in; + dma_addr_t seq_in_ptr; + uint32_t in_ext_length; + struct load_command_s load_dpovrd; + uint32_t dpovrd; +} __attribute__((packed, aligned(64))); + +#endif