Parallelisation (Bash): verschil tussen versies
(Nieuwe pagina aangemaakt met 'WordPress benaderen via de WP-CLI, is vreselijk langzaam. Gelukkig kun je dat zelf in Bash paralleliseren! == Voorbeeld == === Zonder parallelisation === Dit com...') |
|||
Regel 22: | Regel 22: | ||
=== Parallelisatie! === | === Parallelisatie! === | ||
− | In Bash kun je met <code>&</code> aangeven dat het volgende commando kan starten voordat het huidige commando (waar de & bij hoort) beëindigt is. En daarmee blijk je prima te kunnen paralleliseren! | + | In Bash kun je met <code>&</code> aangeven dat het volgende commando kan starten voordat het huidige commando (waar de & bij hoort) beëindigt is. En daarmee blijk je prima te kunnen paralleliseren! Dit is dezelfde code als hierboven, ca. 100x toegepast: |
<pre> | <pre> |
Versie van 27 sep 2021 15:19
WordPress benaderen via de WP-CLI, is vreselijk langzaam. Gelukkig kun je dat zelf in Bash paralleliseren!
Voorbeeld
Zonder parallelisation
Dit commando duurt ca. 5s.:
wp wc shop_order list --user=4 --field=id | xargs -n1 wp wc shop_order delete --user=4 --force=1
Ditzelfde commando op een VPS met 4 ipv 2 CPU's, duurt even lang (zelfs iets langer). Dat was geen verbazing: Dit PHP-commando is niet echt te paralleliseren, want het is één seriele aangelegenheid. Via tops
en ps
kon ik zien dat er wel degelijk aparte processen aan te pas komen. Zoiets als:
- Commando-als-geheel
xargs
php
MySQL
.
Maar nog steeds is er effectief geen sprake van parallelisatie, omdat deze processen welliswaar apart zijn, maar nog steeds seriëel worden doorlopen. Hetzelfde probleem als dat gamers meer geholpen zijn bij hele snelle processoren, dan bij veel processoren. Helaas kan ik bij TransIP niet voor snellere processoren kiezen. Alleen voor meer processoren - Lees verder!
Parallelisatie!
In Bash kun je met &
aangeven dat het volgende commando kan starten voordat het huidige commando (waar de & bij hoort) beëindigt is. En daarmee blijk je prima te kunnen paralleliseren! Dit is dezelfde code als hierboven, ca. 100x toegepast:
# Parallel (2x) ######################################## # date wp wc shop_order list --user=4 --field=id --per_page=50 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=50 --offset=50 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & date
# Parallel (4x) ######################################## # date wp wc shop_order list --user=4 --field=id --per_page=25 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=25 --offset=25 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=25 --offset=50 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=25 --offset=75 | xargs -n1 wp wc shop_order delete --user=4 --force=1 date
# Parallel (8x) ######################################## # date wp wc shop_order list --user=4 --field=id --per_page=12 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=12 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=24 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=36 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=48 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=60 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=72 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=84 | xargs -n1 wp wc shop_order delete --user=4 --force=1 date
# Parallel (16x) ######################################## # date wp wc shop_order list --user=4 --field=id --per_page=12 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=12 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=24 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=36 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=48 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=60 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=72 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=84 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=96 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=108 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=120 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=132 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=144 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=156 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=168 | xargs -n1 wp wc shop_order delete --user=4 --force=1 & wp wc shop_order list --user=4 --field=id --per_page=12 --offset=180 | xargs -n1 wp wc shop_order delete --user=4 --force=1 date