Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tools - Support GNU Parallel #24

Open
BenB196 opened this issue Sep 8, 2023 · 0 comments
Open

Tools - Support GNU Parallel #24

BenB196 opened this issue Sep 8, 2023 · 0 comments

Comments

@BenB196
Copy link

BenB196 commented Sep 8, 2023

Hi, pretty neat set of scripts this project has.

One thing I've noticed is that on relatively large directories with a large number of files, these scripts can occasionally take a good amount of time.

It would be nice if the scripts could leverage GNU Parallel to be able to process multiple files at once.

Taking a script like:

[OK]  All files are UTF-8 encoded.
$ find . -type f    -print0 | xargs -0 -P 2 -n1 grep -Il '' | tr '\n' '\0' | xargs -0 -P 2 -n1  sh -c 'if [ -f "${1}" ]; then awk "/^\xEF\xBB\xBF/ {print FILENAME} {nextfile}" "$1" || true; fi' --

To:

[OK]  All files are UTF-8 encoded.
$ find . -type f    -print0 | parallel 'xargs -0 -P 2 -n1 grep -Il \'\' | tr \'\n\' \'\0\' | xargs -0 -P 2 -n1  sh -c \'if [ -f "${1}" ]; then awk "/^\xEF\xBB\xBF/ {print FILENAME} {nextfile}" "$1" || true; fi\' --'

I don't fully know if this has been something that has been considered, but just a nice thought.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant