AI Hints
Home›Linux Tutorial›Pipes and Sorting
Linux · Lesson 47 of 120

sort and uniq

Ordering and de-duplicating.

⏱ 2 min read▶ Video at 1:00:16🎓 Free
Watch this lesson in the Linux courseWatch on YouTube ↗Full Linux course

Lists are much more useful when they're in order, and without repeats. Sort puts lines in order, and uniq removes repeated lines. Together, they can answer surprisingly clever questions.

sort and uniq
From the video: sort and uniq
  • Order helps
  • Repeats get in the way
  • sort and uniq

Sorting lines

Here's a fruit list, in random order. Sort puts it in alphabetical order. Notice apple appears three times, and banana twice.

Terminal
cat fruit.txt
banana
apple
cherry
apple
banana
apple
date
sort fruit.txt
apple
apple
apple
banana
banana
cherry
date

Removing repeats

Uniq removes repeated lines, but only when they're next to each other. That's why we always sort first. Sort, then uniq, gives each fruit once. Add dash C, and uniq counts how many times each line appeared. Apple three, banana two. And dash R makes sort reverse the order.

Terminal
sort fruit.txt | uniq
apple
banana
cherry
date
sort fruit.txt | uniq -c
      3 apple
      2 banana
      1 cherry
      1 date
sort -r fruit.txt | uniq
date
cherry
banana
apple

Sorting numbers

Sorting numbers needs care. Normal sort compares text, character by character, so nine comes after one hundred, which looks wrong. Dash N sorts by numeric value. Dash K two tells sort to look at the second column, the scores.

Terminal
sort -k 2 scores.txt
priya 100
jo 72
alex 88
kim 9
sam 95
sort -n -k 2 scores.txt
kim 9
jo 72
alex 88
sam 95
priya 100

Summary

Sort orders lines, dash R reverses, dash N sorts numbers. Uniq removes neighbouring repeats, so sort first, and uniq dash C counts them. Next, cutting out columns.

Key takeaways
  • sort, sort -r, sort -n
  • sort | uniq
  • uniq -c counts
Next up: cut — picking out columns