Compression in save_cache should be parallelled
Yz
Note: This is different from https://ideas.circleci.com/cloud-feature-requests/p/savecache-in-parallel
I'm uploading golang's gocache.
I've tried uses
save_cache
directly and uses pigz to compress then pass it to save_cache
. The custom compression was a lot faster than built-in compression in save_cache
.I got 1.7GB go cache.
Custom compression version:
Compression time: about 16s (in
large
ubuntu machine)save_cache
time: about 9sTotal: about 25s
Just
save_cache
version:save_cache
time: about 44s