Igor Kromin |   Consultant. Coder. Blogger. Tinkerer. Gamer.

Awk is a very useful tool for extracting data from text files, all you have to do is tell it which columns you want to extract and you're usually done. However, what if you didn't want to hardcode the column number and needed the ability to define it dynamically? That gets a little be more complicated but luckily there's a way to solve that requirement with bash and awk.

To demonstrate the requirement, lets say we had a data file with 5 columns like so:
 data_file.txt
A1 B1 C1 D1 E1
A2 B2 C2 D2 E2
A3 B3 C3 D3 E3
...


To start lets say you knew you wanted to extract the first and third column. Then your awk command would look something like this:
 Script
awk '{printf "%s\t%s\n", $1, $3}' < data_file.txt




That is very straight forward and works, but lets say now that for various reasons you didn't know which columns you needed to extract ahead of time (or wanted to make your script more flexible). The obvious thing to do here is to define which columns you were interested in as a script variable like so...
 Script
MY_COL1=1
MY_COL2=3
...


How do you use these with awk though? For example the following: `awk '{printf "%s\t%s\n", $$MY_COL1, $$MY_COL2}' < data_file.txt` will fail with this error:
 Error
awk: illegal field $(), name "MY_COL1"
input record number 1, file
source line number 1


The answer is to use awk's -v parameter to assign variables in awk to tell it which column numbers we're interested in. So assuming the above variables have been assigned with the column numbers we want, the following script will do the trick:
 Script
MY_COL1=1
MY_COL2=3
awk -v c1=$MY_COL1 -v c2=$MY_COL2 '{printf "%s\t%s\n", $c1, $c2}' < data_file.txt


Note how the $1 and $3 in the original awk program are now $c1 and $c2 respectively and both of c1 and c2 variables are assigned values from the shell script variables using the -v parameter.

So now you can redefine which columns are extracted by changing the shell script variable while leaving the awk command alone.

-i

Please leave your comments or feedback below!
comments powered by Disqus
Other posts you may like...
Hi! You can search my blog here ⤵
Or browse the recent top tags...

Recent Blog Posts

Review of Stellar Phoenix Photo Recovery software for Mac

A year later and GitLab still doesn't allow to show private project activity

How to remove caked on stickers from old game carts

How to pass parameters to your PHP script via the command line

jPhotoFrame new layout engine explained with examples

Atari Lynx Multi SD Card cartridge review

Jersey JAX-RS filters and interceptors execution order when throwing Exceptions

Why I'll never renew a domain with GoDaddy again

Fake AirPods - HBQ-i7 TWS wireless headset teardown

How to fix Postman error - Failed to import data: format not recognized

Recent Galleries

BMB-012 Nanoblock T-Rex Skeleton Model assembly

Tiny Arcade revision 6 kit assembly and decal application

Atari Lynx repair - Part 5 - McWill LED screen mod installation

Atari Lynx repair - Part 4 - screen cover replacement

Atari Lynx repair - Part 2 - re-capping the motherboard

Atari Lynx repair - Part 3 - broken speaker replacement

Atari Lynx repair - Part 1 - introduction and case disassembly

Building a custom Atari Lynx game box storage shelf unit in a day

Protecting old Atari Lynx game boxes with snug fit plastic sleeves

Monument Valley 2 is released and does not disappoint

Blogs and Friends

Matt Moores Blog
Georgi's FlatPress Guide
Perplexing Permutations
The Security Sleuth
Ilia Rogatchevski
Travelling Fairy

Blog Activity

Blog Activity