Parallelisation (Bash)

WordPress benaderen via de WP-CLI, is vreselijk langzaam. Gelukkig kun je dat zelf in Bash paralleliseren!

Voorbeeld

Zonder parallelisation

Dit commando duurt ca. 5s.:

wp wc shop_order list --user=4 --field=id | xargs -n1 wp wc shop_order delete --user=4 --force=1

Ditzelfde commando op een VPS met 4 ipv 2 CPU's, duurt even lang (zelfs iets langer). Dat was geen verbazing: Dit PHP-commando is niet echt te paralleliseren, want het is één seriele aangelegenheid. Via tops en ps kon ik zien dat er wel degelijk aparte processen aan te pas komen. Zoiets als:

• Commando-als-geheel
• xargs
• php
• MySQL.

Maar nog steeds is er effectief geen sprake van parallelisatie, omdat deze processen welliswaar apart zijn, maar nog steeds seriëel worden doorlopen. Hetzelfde probleem als dat gamers meer geholpen zijn bij hele snelle processoren, dan bij veel processoren. Helaas kan ik bij TransIP niet voor snellere processoren kiezen. Alleen voor meer processoren - Lees verder!

Parallelisatie!

In Bash kun je met & aangeven dat het volgende commando kan starten voordat het huidige commando (waar de & bij hoort) beëindigt is. En daarmee blijk je prima te kunnen paralleliseren! Dit is dezelfde code als hierboven, ca. 100x toegepast:

# Parallel (2x)
########################################
#
date
	wp wc shop_order list --user=4 --field=id --per_page=50             | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=50 --offset=50 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
date

# Parallel (4x)
########################################
#
date
	wp wc shop_order list --user=4 --field=id --per_page=25             | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=25 --offset=25 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=25 --offset=50 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=25 --offset=75 | xargs -n1 wp wc shop_order delete --user=4 --force=1	
date

# Parallel (8x)
########################################
#
date
	wp wc shop_order list --user=4 --field=id --per_page=12             | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=12 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=24 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=36 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &	
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=48 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=60 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=72 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=84 | xargs -n1 wp wc shop_order delete --user=4 --force=1	
date

# Parallel (16x)
########################################
#
date
	wp wc shop_order list --user=4 --field=id --per_page=12              | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=12  | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=24  | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=36  | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=48  | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=60  | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=72  | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=84  | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=96  | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=108 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=120 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=132 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=144 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=156 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=168 | xargs -n1 wp wc shop_order delete --user=4 --force=1 &
	wp wc shop_order list --user=4 --field=id --per_page=12 --offset=180 | xargs -n1 wp wc shop_order delete --user=4 --force=1
date