Use awk to split a text file into multiple files based on some criteria

In the spirit of “when you learn something stick it in a blog post so you can find it later” here’s a solution to splitting a file into multiple files using the Unix/Linux command awk.

Here’s a demo script that dumps some sample data into a file and then uses awk to split it into multiple output streams:

echo "John,Bananas,London,10" > /tmp/input
echo "John,Apples,London,5" >> /tmp/input
echo "Jane,Bananas,London,2" >> /tmp/input
echo "John,Apples,Leeds,3" >> /tmp/input
echo "Jane,Bananas,Leeds,8" >> /tmp/input
echo
echo Jane buying Bananas
echo ===================
cat /tmp/input | awk -F, '$1=="Jane" && $2=="Bananas"'
echo
echo
echo John buying in London
echo =====================
cat /tmp/input | awk -F, '$1=="John" && $3=="London"'
echo
echo
echo Buying in Leeds
echo ===============
cat /tmp/input | awk -F, '$3=="Leeds"'
echo
echo

which when run returns this output:

$ . /tmp/demoawk.sh

Jane buying Bananas
===================
Jane,Bananas,London,2
Jane,Bananas,Leeds,8

John buying in London
=====================
John,Bananas,London,10
John,Apples,London,5

Buying in Leeds
===============
John,Apples,Leeds,3
Jane,Bananas,Leeds,8

Like I said, I wrote this for my own reference but if someone else finds it useful, so much the better! Thank you to my colleague Marcin Kulisz who helped me out with this.

JT