From patchwork Mon Jan 30 10:21:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ferruh Yigit X-Patchwork-Id: 20071 Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 5BE125911; Mon, 30 Jan 2017 11:21:24 +0100 (CET) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id B44CCFFA for ; Mon, 30 Jan 2017 11:21:19 +0100 (CET) Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP; 30 Jan 2017 02:21:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,311,1477983600"; d="scan'208";a="53715009" Received: from sivswdev02.ir.intel.com ([10.237.217.46]) by orsmga004.jf.intel.com with ESMTP; 30 Jan 2017 02:21:17 -0800 From: Ferruh Yigit To: Thomas Monjalon Cc: dev@dpdk.org, Ferruh Yigit Date: Mon, 30 Jan 2017 10:21:15 +0000 Message-Id: <20170130102115.20975-1-ferruh.yigit@intel.com> X-Mailer: git-send-email 2.8.4 In-Reply-To: <20170122015034.19824-1-ferruh.yigit@intel.com> References: <20170122015034.19824-1-ferruh.yigit@intel.com> Subject: [dpdk-dev] [PATCH v2] mk: parallelize make config X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" make config dependency resolving was always running serial, parallelize it for better performance. $ time make T=x86_64-native-linuxapp-gcc config real 0m12.633s $ time make -j8 T=x86_64-native-linuxapp-gcc config real 0m1.826s When config creation done under a single make target, using a for loop, make has no control on the action, and it needs to run as implemented in the rule. But if for loop converted into multiple targets, make can detect independent targets and run them parallel based on -j parameter. Signed-off-by: Ferruh Yigit Acked-by: Thomas Monjalon --- devtools/test-build.sh | 2 +- mk/internal/rte.depdirs-post.mk | 11 ++++++----- mk/rte.sdkdepdirs.mk | 21 ++++++++++----------- mk/rte.subdir.mk | 17 +++++++---------- 4 files changed, 24 insertions(+), 27 deletions(-) diff --git a/devtools/test-build.sh b/devtools/test-build.sh index 680d79b..0f131fc 100755 --- a/devtools/test-build.sh +++ b/devtools/test-build.sh @@ -146,7 +146,7 @@ config () # fi if [ ! -e $1/.config ] || $reconfig ; then echo "================== Configure $1" - make T=$2 O=$1 config + make -j$J T=$2 O=$1 config echo 'Customize configuration' # Built-in options (lowercase) diff --git a/mk/internal/rte.depdirs-post.mk b/mk/internal/rte.depdirs-post.mk index 102a369..eb73ad3 100644 --- a/mk/internal/rte.depdirs-post.mk +++ b/mk/internal/rte.depdirs-post.mk @@ -29,11 +29,12 @@ # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -.PHONY: depdirs -depdirs: - @for d in $(DEPDIRS-y); do \ - $(RTE_SDK)/buildtools/depdirs-rule.sh $(S) $$d ; \ - done +.PHONY: depdirs $(DEPDIRS-y) +depdirs: $(DEPDIRS-y) + @echo "" + +$(DEPDIRS-y): + @$(RTE_SDK)/buildtools/depdirs-rule.sh $(S) $@ .PHONY: depgraph depgraph: diff --git a/mk/rte.sdkdepdirs.mk b/mk/rte.sdkdepdirs.mk index bebaf2a..38fd863 100644 --- a/mk/rte.sdkdepdirs.mk +++ b/mk/rte.sdkdepdirs.mk @@ -36,19 +36,18 @@ ifeq (,$(wildcard $(RTE_OUTPUT)/Makefile)) $(error "need a make config first") endif -# use a "for" in a shell to process dependencies: we don't want this -# task to be run in parallel. +DEPDIR_FILES = $(addsuffix /.depdirs, $(addprefix $(BUILDDIR)/,$(ROOTDIRS-y))) + .PHONY: depdirs depdirs: $(RTE_OUTPUT)/.depdirs -$(RTE_OUTPUT)/.depdirs: $(RTE_OUTPUT)/.config - @rm -f $(RTE_OUTPUT)/.depdirs ; \ - for d in $(ROOTDIRS-y); do \ - if [ -f $(RTE_SRCDIR)/$$d/Makefile ]; then \ - [ -d $(BUILDDIR)/$$d ] || mkdir -p $(BUILDDIR)/$$d ; \ - $(MAKE) S=$$d -f $(RTE_SRCDIR)/$$d/Makefile depdirs \ - >> $(RTE_OUTPUT)/.depdirs ; \ - fi ; \ - done +$(RTE_OUTPUT)/.depdirs: $(DEPDIR_FILES) + @rm -f $@ + @sort -u -o $@ $(DEPDIR_FILES) + +$(DEPDIR_FILES): $(RTE_OUTPUT)/.config + @dir=$(notdir $(@D)); \ + [ -d $(BUILDDIR)/$$dir ] || mkdir -p $(BUILDDIR)/$$dir; \ + $(MAKE) S=$$dir -f $(RTE_SRCDIR)/$$dir/Makefile depdirs > $@ .PHONY: depgraph depgraph: diff --git a/mk/rte.subdir.mk b/mk/rte.subdir.mk index 256e64e..5341f1f 100644 --- a/mk/rte.subdir.mk +++ b/mk/rte.subdir.mk @@ -76,7 +76,7 @@ clean: _postclean # include .depdirs and define rules to order priorities between build # of directories. # -include $(RTE_OUTPUT)/.depdirs +-include $(RTE_OUTPUT)/.depdirs define depdirs_rule $(1): $(sort $(patsubst $(S)/%,%,$(LOCAL_DEPDIRS-$(S)/$(1)))) @@ -84,16 +84,13 @@ endef $(foreach d,$(DIRS-y),$(eval $(call depdirs_rule,$(d)))) +DEPDIRS = $(wildcard $(addprefix $(S)/,$(DIRS-y))) -# use a "for" in a shell to process dependencies: we don't want this -# task to be run in parallel. -.PHONY: depdirs -depdirs: - @for d in $(DIRS-y); do \ - if [ -f $(SRCDIR)/$$d/Makefile ]; then \ - $(MAKE) S=$S/$$d -f $(SRCDIR)/$$d/Makefile depdirs ; \ - fi ; \ - done +.PHONY: depdirs $(DEPDIRS) +depdirs: $(DEPDIRS) + +$(DEPDIRS): + @$(MAKE) S=$@ -f $(RTE_SRCDIR)/$@/Makefile depdirs .PHONY: depgraph depgraph: