Message ID | 20230929083443.9925-1-pbhagavatula@marvell.com (mailing list archive) |
---|---|
State | Superseded |
Headers |
Return-Path: <ci-bounces@dpdk.org> X-Original-To: patchwork@inbox.dpdk.org Delivered-To: patchwork@inbox.dpdk.org Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E65E142671; Fri, 29 Sep 2023 10:35:33 +0200 (CEST) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id E014E402E7; Fri, 29 Sep 2023 10:35:33 +0200 (CEST) Received: from mx0b-0016f401.pphosted.com (mx0a-0016f401.pphosted.com [67.231.148.174]) by mails.dpdk.org (Postfix) with ESMTP id D95BD402E6 for <ci@dpdk.org>; Fri, 29 Sep 2023 10:35:31 +0200 (CEST) Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 38T273XA031700; Fri, 29 Sep 2023 01:35:29 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding : content-type; s=pfpt0220; bh=yl3sd625AriMvvdsXO/GmAgPl4fIO2CEoaGZxumgH4c=; b=LRmZECENeQFfR8aw6RP07YqFXvzntdN9paSOuuCm/7/JPZOwGdJ/bPoYM30apezwAjKg RX1aHEv/fMFfyjWQgfSZxJzKqks7X+ogwCfT5irdQjBsVmCctm+j4Bj5+/O5gowHu+xx fok+SlJTCkGs9mMiOyZ5SGnFsyHk54Fo8OXwqZ1TwfPbrnwz8lqokzu5o9ShzU5rkE95 Qe0oP43Wv/Up4ym62g+6GeO2Lxer8MZbRPXfxF5NOuhQPqXA4uI4YZ8OKnWL99aaQFPb Kj/hIAQ0ynH6lDSEBXSM8yod2TBCE/BcZxif6QHPVFSIyLKIlpyE9vZLSDg3ijodwRhF Xg== Received: from dc5-exch02.marvell.com ([199.233.59.182]) by mx0a-0016f401.pphosted.com (PPS) with ESMTPS id 3td7y6vgek-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Fri, 29 Sep 2023 01:35:27 -0700 Received: from DC5-EXCH02.marvell.com (10.69.176.39) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server (TLS) id 15.0.1497.48; Fri, 29 Sep 2023 01:34:51 -0700 Received: from maili.marvell.com (10.69.176.80) by DC5-EXCH02.marvell.com (10.69.176.39) with Microsoft SMTP Server id 15.0.1497.48 via Frontend Transport; Fri, 29 Sep 2023 01:34:51 -0700 Received: from MININT-80QBFE8.corp.innovium.com (MININT-80QBFE8.marvell.com [10.28.164.106]) by maili.marvell.com (Postfix) with ESMTP id 102EA3F7073; Fri, 29 Sep 2023 01:34:48 -0700 (PDT) From: <pbhagavatula@marvell.com> To: <jerinj@marvell.com>, <alialnu@nvidia.com>, <aconole@redhat.com> CC: <ci@dpdk.org>, Pavan Nikhilesh <pbhagavatula@marvell.com> Subject: [PATCH] pw_maintainers_cli: enhance tree selection Date: Fri, 29 Sep 2023 14:04:43 +0530 Message-ID: <20230929083443.9925-1-pbhagavatula@marvell.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-Proofpoint-ORIG-GUID: heITfEAuwDCzYaQqRvZTs9PcCv2VKjsL X-Proofpoint-GUID: heITfEAuwDCzYaQqRvZTs9PcCv2VKjsL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-09-29_07,2023-09-28_03,2023-05-22_02 X-BeenThere: ci@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK CI discussions <ci.dpdk.org> List-Unsubscribe: <https://mails.dpdk.org/options/ci>, <mailto:ci-request@dpdk.org?subject=unsubscribe> List-Archive: <http://mails.dpdk.org/archives/ci/> List-Post: <mailto:ci@dpdk.org> List-Help: <mailto:ci-request@dpdk.org?subject=help> List-Subscribe: <https://mails.dpdk.org/listinfo/ci>, <mailto:ci-request@dpdk.org?subject=subscribe> Errors-To: ci-bounces@dpdk.org |
Series |
pw_maintainers_cli: enhance tree selection
|
|
Commit Message
Pavan Nikhilesh Bhagavatula
Sept. 29, 2023, 8:34 a.m. UTC
From: Pavan Nikhilesh <pbhagavatula@marvell.com> When longest prefix match doesnt find a suitable tree, pick the tree which has the highest modified file count instead of defauting to main tree. Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> --- tools/pw_maintainers_cli.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Comments
> -----Original Message----- > From: pbhagavatula@marvell.com <pbhagavatula@marvell.com> > Sent: Friday, September 29, 2023 2:05 PM > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; alialnu@nvidia.com; > aconole@redhat.com > Cc: ci@dpdk.org; Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com> > Subject: [PATCH] pw_maintainers_cli: enhance tree selection > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > When longest prefix match doesnt find a suitable tree, pick the > tree which has the highest modified file count instead of defauting > to main tree. > This change is need to find the correct branch when a patch has a specification Change followed by a implementation of driver layer example: https://patches.dpdk.org/project/dpdk/list/?series=29675 > Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com> > --- > tools/pw_maintainers_cli.py | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/tools/pw_maintainers_cli.py b/tools/pw_maintainers_cli.py > index c7b5ba0..4aa46ae 100755 > --- a/tools/pw_maintainers_cli.py > +++ b/tools/pw_maintainers_cli.py > @@ -46,6 +46,7 @@ import re > import argparse > import fnmatch > > +from collections import Counter > from requests.exceptions import HTTPError > > from git_pw import config > @@ -276,6 +277,7 @@ class Maintainers(object): > dpdk-next-crypto + dpdk-next-virtio = dpdk > dpdk-next-net-intel + dpdk-next-net-mlx = dpdk-next-net > """ > + highest_tree = Counter(tree_list).most_common(1)[0][0] > # Make sure the list is unique. > tree_list = list(set(tree_list)) > > @@ -287,7 +289,9 @@ class Maintainers(object): > os.path.commonprefix(_tree_list).rstrip('-').replace( > 'dpdk-next-net-virtio', 'dpdk-next-virtio') > # There is no 'dpdk-next' named tree. > - if common_prefix.endswith('dpdk-next') or > common_prefix.endswith('/'): > + if common_prefix.endswith('dpdk-next'): > + common_prefix = highest_tree > + elif common_prefix.endswith('/'): > common_prefix = 'git://dpdk.org/dpdk' > return common_prefix > > -- > 2.25.1
On Fri, Sep 29, 2023 at 11:21 AM Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com> wrote: > > -----Original Message----- > > From: pbhagavatula@marvell.com <pbhagavatula@marvell.com> > > Sent: Friday, September 29, 2023 2:05 PM > > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; alialnu@nvidia.com; > > aconole@redhat.com > > Cc: ci@dpdk.org; Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com> > > Subject: [PATCH] pw_maintainers_cli: enhance tree selection > > > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > > > When longest prefix match doesnt find a suitable tree, pick the > > tree which has the highest modified file count instead of defauting > > to main tree. > > This script helps with the delegation but nothing prevents submitters or maintainers from updating this in patchwork themselves. I did not test it but this heuristic seems complex in how it is affected by existing git history. It makes this script harder to understand / predict the outcome. I fear side effects. > > This change is need to find the correct branch when a patch has a specification > Change followed by a implementation of driver layer example: This should be in the commitlog from the start as it exposes the need, rather than the solution implemented by the patch. > > https://patches.dpdk.org/project/dpdk/list/?series=29675
> On Fri, Sep 29, 2023 at 11:21 AM Pavan Nikhilesh Bhagavatula > <pbhagavatula@marvell.com> wrote: > > > -----Original Message----- > > > From: pbhagavatula@marvell.com <pbhagavatula@marvell.com> > > > Sent: Friday, September 29, 2023 2:05 PM > > > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; alialnu@nvidia.com; > > > aconole@redhat.com > > > Cc: ci@dpdk.org; Pavan Nikhilesh Bhagavatula > <pbhagavatula@marvell.com> > > > Subject: [PATCH] pw_maintainers_cli: enhance tree selection > > > > > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > > > > > When longest prefix match doesnt find a suitable tree, pick the > > > tree which has the highest modified file count instead of defauting > > > to main tree. > > > > > This script helps with the delegation but nothing prevents submitters > or maintainers from updating this in patchwork themselves. > The script is used by CI to decide which branch to apply the patches on and run the tests. For example in the CI run http://mails.dpdk.org/archives/test-report/2023-September/468555.html the script incorrectly picks dpdk main tree instead of dpdk-next-event and fails. > I did not test it but this heuristic seems complex in how it is > affected by existing git history. > It makes this script harder to understand / predict the outcome. > I fear side effects. > It is not based on git history, It picks the tree based on the files that were changed in the patch. Also, it only effects the case where its not able to find the common tree. > > > > > This change is need to find the correct branch when a patch has a > specification > > Change followed by a implementation of driver layer example: > > This should be in the commitlog from the start as it exposes the need, > rather than the solution implemented by the patch. > I can change this in the next version. > > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patches.dpdk.org_project_dpdk_list_-3Fseries- > 3D29675&d=DwIFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=E3SgYMjtKCMVsB- > fmvgGV3o- > g_fjLhk5Pupi9ijohpc&m=QbsEVS0vy8B7ZVY0WPAMjlHbfOxASgG_XP4P7_pri > aunQeURVo3SJaIvt3GsjfMe&s=kdARtlZA5zKbOuliw3uJFntF- > IjXGY8OXli9SkD2rDg&e= > > > > -- > David Marchand
29/09/2023 11:21, Pavan Nikhilesh Bhagavatula: > > > -----Original Message----- > > From: pbhagavatula@marvell.com <pbhagavatula@marvell.com> > > Sent: Friday, September 29, 2023 2:05 PM > > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; alialnu@nvidia.com; > > aconole@redhat.com > > Cc: ci@dpdk.org; Pavan Nikhilesh Bhagavatula <pbhagavatula@marvell.com> > > Subject: [PATCH] pw_maintainers_cli: enhance tree selection > > > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > > > When longest prefix match doesnt find a suitable tree, pick the > > tree which has the highest modified file count instead of defauting > > to main tree. > > > > This change is need to find the correct branch when a patch has a specification > Change followed by a implementation of driver layer example: > > https://patches.dpdk.org/project/dpdk/list/?series=29675 That's expected: when a series touches more than a tree scope, it goes to main. But that's not the issue here. Both eventdev lib, test and drivers belong to the eventdev tree. So why it is not already delegated to eventdev? Please dig more.
> > > > > -----Original Message----- > > > From: pbhagavatula@marvell.com <pbhagavatula@marvell.com> > > > Sent: Friday, September 29, 2023 2:05 PM > > > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; alialnu@nvidia.com; > > > aconole@redhat.com > > > Cc: ci@dpdk.org; Pavan Nikhilesh Bhagavatula > <pbhagavatula@marvell.com> > > > Subject: [PATCH] pw_maintainers_cli: enhance tree selection > > > > > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > > > > > When longest prefix match doesnt find a suitable tree, pick the > > > tree which has the highest modified file count instead of defauting > > > to main tree. > > > > > > > This change is need to find the correct branch when a patch has a > specification > > Change followed by a implementation of driver layer example: > > > > https://urldefense.proofpoint.com/v2/url?u=https- > 3A__patches.dpdk.org_project_dpdk_list_-3Fseries- > 3D29675&d=DwICAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=E3SgYMjtKCMVsB- > fmvgGV3o-g_fjLhk5Pupi9ijohpc&m=WKN1erKSjH9MuEyyRs- > R3jYC5geZeUeNipx2GQlQpQSuQQCet70Torr1oSdZNvZp&s=6XFs2Ggg5G0VFd > YEaRexumEXM5JKc2vs5dmrRNcIUZw&e= > > That's expected: when a series touches more than a tree scope, > it goes to main. > But that's not the issue here. > Both eventdev lib, test and drivers belong to the eventdev tree. > So why it is not already delegated to eventdev? > Please dig more. > > The main issue is the driver implementation touches common/cnxk which ties it to next-net-mrvl tree which causes the conflict. We have few options here, 1. Based on max number of files touched per tree (Current patch) 2. Ignore driver/common from tree selection when there is a conflict with other tree. 3. When there is a conflict between trees choose the common tree instead of company specific tree Example, dpdk-next-eventdev + dpdk-next-net-mrvl = dpdk-next-eventdev
29/09/2023 12:54, Pavan Nikhilesh Bhagavatula: > > > From: pbhagavatula@marvell.com <pbhagavatula@marvell.com> > > > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > > > > > > > When longest prefix match doesnt find a suitable tree, pick the > > > > tree which has the highest modified file count instead of defauting > > > > to main tree. > > > > > > > > > > This change is need to find the correct branch when a patch has a > > specification > > > Change followed by a implementation of driver layer example: > > > > > > https://urldefense.proofpoint.com/v2/url?u=https- > > 3A__patches.dpdk.org_project_dpdk_list_-3Fseries- > > 3D29675&d=DwICAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=E3SgYMjtKCMVsB- > > fmvgGV3o-g_fjLhk5Pupi9ijohpc&m=WKN1erKSjH9MuEyyRs- > > R3jYC5geZeUeNipx2GQlQpQSuQQCet70Torr1oSdZNvZp&s=6XFs2Ggg5G0VFd > > YEaRexumEXM5JKc2vs5dmrRNcIUZw&e= > > > > That's expected: when a series touches more than a tree scope, > > it goes to main. > > But that's not the issue here. > > Both eventdev lib, test and drivers belong to the eventdev tree. > > So why it is not already delegated to eventdev? > > Please dig more. > > > > > > The main issue is the driver implementation touches common/cnxk which ties it to > next-net-mrvl tree which causes the conflict. > > We have few options here, > 1. Based on max number of files touched per tree (Current patch) > 2. Ignore driver/common from tree selection when there is a conflict with other tree. > 3. When there is a conflict between trees choose the common tree instead of company specific tree > Example, dpdk-next-eventdev + dpdk-next-net-mrvl = dpdk-next-eventdev I have a preference for option 2.
> -----Original Message----- > From: Thomas Monjalon <thomas@monjalon.net> > Sent: Friday, September 29, 2023 4:40 PM > To: Jerin Jacob Kollanukkaran <jerinj@marvell.com>; alialnu@nvidia.com; > aconole@redhat.com; ci@dpdk.org; Pavan Nikhilesh Bhagavatula > <pbhagavatula@marvell.com> > Subject: Re: [EXT] Re: [PATCH] pw_maintainers_cli: enhance tree selection > > 29/09/2023 12:54, Pavan Nikhilesh Bhagavatula: > > > > From: pbhagavatula@marvell.com <pbhagavatula@marvell.com> > > > > > From: Pavan Nikhilesh <pbhagavatula@marvell.com> > > > > > > > > > > When longest prefix match doesnt find a suitable tree, pick the > > > > > tree which has the highest modified file count instead of > > > > > defauting to main tree. > > > > > > > > > > > > > This change is need to find the correct branch when a patch has a > > > specification > > > > Change followed by a implementation of driver layer example: > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=https- > > > 3A__patches.dpdk.org_project_dpdk_list_-3Fseries- > > > 3D29675&d=DwICAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=E3SgYMjtKCMVsB- > > > fmvgGV3o-g_fjLhk5Pupi9ijohpc&m=WKN1erKSjH9MuEyyRs- > > > > R3jYC5geZeUeNipx2GQlQpQSuQQCet70Torr1oSdZNvZp&s=6XFs2Ggg5G0VFd > > > YEaRexumEXM5JKc2vs5dmrRNcIUZw&e= > > > > > > That's expected: when a series touches more than a tree scope, it > > > goes to main. > > > But that's not the issue here. > > > Both eventdev lib, test and drivers belong to the eventdev tree. > > > So why it is not already delegated to eventdev? > > > Please dig more. > > > > > > > > > > The main issue is the driver implementation touches common/cnxk which > > ties it to next-net-mrvl tree which causes the conflict. > > > > We have few options here, > > 1. Based on max number of files touched per tree (Current patch) 2. > > Ignore driver/common from tree selection when there is a conflict with other > tree. > > 3. When there is a conflict between trees choose the common tree instead of > company specific tree > > Example, dpdk-next-eventdev + dpdk-next-net-mrvl = > > dpdk-next-eventdev > > I have a preference for option 2. +1 for option 2. > > >
diff --git a/tools/pw_maintainers_cli.py b/tools/pw_maintainers_cli.py index c7b5ba0..4aa46ae 100755 --- a/tools/pw_maintainers_cli.py +++ b/tools/pw_maintainers_cli.py @@ -46,6 +46,7 @@ import re import argparse import fnmatch +from collections import Counter from requests.exceptions import HTTPError from git_pw import config @@ -276,6 +277,7 @@ class Maintainers(object): dpdk-next-crypto + dpdk-next-virtio = dpdk dpdk-next-net-intel + dpdk-next-net-mlx = dpdk-next-net """ + highest_tree = Counter(tree_list).most_common(1)[0][0] # Make sure the list is unique. tree_list = list(set(tree_list)) @@ -287,7 +289,9 @@ class Maintainers(object): os.path.commonprefix(_tree_list).rstrip('-').replace( 'dpdk-next-net-virtio', 'dpdk-next-virtio') # There is no 'dpdk-next' named tree. - if common_prefix.endswith('dpdk-next') or common_prefix.endswith('/'): + if common_prefix.endswith('dpdk-next'): + common_prefix = highest_tree + elif common_prefix.endswith('/'): common_prefix = 'git://dpdk.org/dpdk' return common_prefix