Archive for July 2017
Use awk to split a text file into multiple files based on some criteria
In the spirit of “when you learn something stick it in a blog post so you can find it later” here’s a solution to splitting a file into multiple files using the Unix/Linux command awk.
Here’s a demo script that dumps some sample data into a file and then uses awk to split it into multiple output streams:
echo "John,Bananas,London,10" > /tmp/input echo "John,Apples,London,5" >> /tmp/input echo "Jane,Bananas,London,2" >> /tmp/input echo "John,Apples,Leeds,3" >> /tmp/input echo "Jane,Bananas,Leeds,8" >> /tmp/input echo echo Jane buying Bananas echo =================== cat /tmp/input | awk -F, '$1=="Jane" && $2=="Bananas"' echo echo echo John buying in London echo ===================== cat /tmp/input | awk -F, '$1=="John" && $3=="London"' echo echo echo Buying in Leeds echo =============== cat /tmp/input | awk -F, '$3=="Leeds"' echo echo
which when run returns this output:
$ . /tmp/demoawk.sh Jane buying Bananas =================== Jane,Bananas,London,2 Jane,Bananas,Leeds,8 John buying in London ===================== John,Bananas,London,10 John,Apples,London,5 Buying in Leeds =============== John,Apples,Leeds,3 Jane,Bananas,Leeds,8
Like I said, I wrote this for my own reference but if someone else finds it useful, so much the better! Thank you to my colleague Marcin Kulisz who helped me out with this.
JT