sort and uniq
Ordering and de-duplicating.
Lists are much more useful when they're in order, and without repeats. Sort puts lines in order, and uniq removes repeated lines. Together, they can answer surprisingly clever questions.

- Order helps
- Repeats get in the way
- sort and uniq
Sorting lines
Here's a fruit list, in random order. Sort puts it in alphabetical order. Notice apple appears three times, and banana twice.
cat fruit.txt banana apple cherry apple banana apple date sort fruit.txt apple apple apple banana banana cherry date
Removing repeats
Uniq removes repeated lines, but only when they're next to each other. That's why we always sort first. Sort, then uniq, gives each fruit once. Add dash C, and uniq counts how many times each line appeared. Apple three, banana two. And dash R makes sort reverse the order.
sort fruit.txt | uniq apple banana cherry date sort fruit.txt | uniq -c 3 apple 2 banana 1 cherry 1 date sort -r fruit.txt | uniq date cherry banana apple
Sorting numbers
Sorting numbers needs care. Normal sort compares text, character by character, so nine comes after one hundred, which looks wrong. Dash N sorts by numeric value. Dash K two tells sort to look at the second column, the scores.
sort -k 2 scores.txt priya 100 jo 72 alex 88 kim 9 sam 95 sort -n -k 2 scores.txt kim 9 jo 72 alex 88 sam 95 priya 100
Summary
Sort orders lines, dash R reverses, dash N sorts numbers. Uniq removes neighbouring repeats, so sort first, and uniq dash C counts them. Next, cutting out columns.
Key takeaways- sort, sort -r, sort -n
- sort | uniq
- uniq -c counts