{"id":6925,"date":"2018-12-29T22:10:29","date_gmt":"2018-12-29T22:10:29","guid":{"rendered":"https:\/\/www.appservgrid.com\/paw92\/?p=6925"},"modified":"2018-12-30T05:16:19","modified_gmt":"2018-12-30T05:16:19","slug":"bash-uniq-command-linux-hint","status":"publish","type":"post","link":"https:\/\/www.appservgrid.com\/paw92\/index.php\/2018\/12\/29\/bash-uniq-command-linux-hint\/","title":{"rendered":"Bash uniq Command \u2013 Linux Hint"},"content":{"rendered":"<p>Linux users need to create or read the text file in regular basis for many purposes. A text file can contain different types of numeric and character data. Same data can be stored multiple times in a text file. Sometimes, you may require reading any text file by omitting duplicate lines of data. Bash<\/p>\n<p>uniq<\/p>\n<p>command is a useful command line utility tool that is used to read a text file by filtering or removing adjacent duplicate lines from the text file.<\/p>\n<p>uniq<\/p>\n<p>command is used to detect the adjacent lines from a file and write the content of the file by filtering the duplicate values or write only the duplicate lines into another file.<\/p>\n<p>uniq [OPTION] [ INPUT [OUTPUT] ]<\/p>\n<p>Here, OPTION, INPUT, and OUTPUT are optional. If you use only uniq command without any option or input\/output filename then this command will apply on the standard input data. Many types of options can be used with this command to filter duplicate data in various ways from any text file. If you use an input file name with this command then the data will filter from that file. If you execute the command with the option, input filename, and output filename then the data will filter from input file based on the option and write the output into the output file.<\/p>\n<h2>Options:<\/h2>\n<p>Some major options of uniq command are discussed below.<\/p>\n<ul>\n<li>-f N or \u2013skip-fields=N<\/li>\n<\/ul>\n<p>It is used to skip N fields before detecting the uniqueness of data. Fields are the group of characters separated by whitespace or tab.<\/p>\n<ul>\n<li>-s N or \u2013skip-chars=N<\/li>\n<\/ul>\n<p>It is used to skip N characters before detecting the uniqueness of data.<\/p>\n<ul>\n<li>-w N or \u2013check-chars=N<\/li>\n<\/ul>\n<p>It is used to compare N characters only in a line.<\/p>\n<ul>\n<li>-c or \u2013count<\/li>\n<\/ul>\n<p>It is used to count how many times a line repeated in the searching data and the values are shown as the prefix of that line.<\/p>\n<ul>\n<li>-z or \u2013zero-terminated<\/li>\n<\/ul>\n<p>It is used to terminate the line with 0 bytes instead of using newline.<\/p>\n<ul>\n<li>-d or \u2013repeated<\/li>\n<\/ul>\n<p>It is used to print all repeated lines only.<\/p>\n<ul>\n<li>-D or \u2013all-repeated[=METHOD]<\/li>\n<\/ul>\n<p>It is used to print all repeated lines based on the used method. The following methods can be used with this option.<\/p>\n<p>none: It is the default method and doesn\u2019t delimit duplicate lines.<br \/>\nprepend: It adds a blank line before each set of duplicate lines.<br \/>\nseparate: It adds a blank line between two duplicate lines.<\/p>\n<ul>\n<li>-u or \u2013unique<\/li>\n<\/ul>\n<p>It is used to print the unique lines only.<\/p>\n<ul>\n<li>-i or \u2013ignore-case<\/li>\n<\/ul>\n<p>It is used for case-insensitive comparison.<\/p>\n<h3>Examples of uniq command<\/h3>\n<p>Create a text file named uniq_test.txt with the following content:<\/p>\n<p>Bash Programming<br \/>\nBash Programming<br \/>\nPython Programming<br \/>\nI like PHP Programming<br \/>\nI like Java Programming<\/p>\n<h3>Example#1: Using -f option<\/h3>\n<p>The following command will apply uniq command by skipping first two fields of each line from uniq_test.txt file.<\/p>\n<p>$ uniq -f 2 uniq_test.txt<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/linuxhint.com\/wp-content\/uploads\/2018\/12\/1-40.png\" alt=\"\" width=\"734\" height=\"257\" \/><\/p>\n<h3>Example#2: Using -s option<\/h3>\n<p>The following command will apply uniq command by skipping 4 characters from each line of uniq_test.txt file.<\/p>\n<p>$ uniq -s 4 uniq_test.txt<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/linuxhint.com\/wp-content\/uploads\/2018\/12\/2-36.png\" alt=\"\" width=\"734\" height=\"175\" \/><\/p>\n<h3>Example#3: Using \u2013w option<\/h3>\n<p>The following command will apply uniq command by comparing the first two characters of each line.<\/p>\n<p>$ uniq -w 2 uniq_test.txt<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/linuxhint.com\/wp-content\/uploads\/2018\/12\/3-35.png\" alt=\"\" width=\"728\" height=\"150\" \/><\/p>\n<h3>Example#4: Using \u2013c option<\/h3>\n<p>The following command will count the appearance of each line in the file and displays the number at the front of each line of the output.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/linuxhint.com\/wp-content\/uploads\/2018\/12\/4-34.png\" alt=\"\" width=\"731\" height=\"162\" \/><\/p>\n<h3>Example#5: Using \u2013d option<\/h3>\n<p>The following command displays those lines from the file only that appeared multiple times in the file. Only one line has appeared two times in uniq_test.txt file which is displayed as output.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/linuxhint.com\/wp-content\/uploads\/2018\/12\/5-36.png\" alt=\"\" width=\"734\" height=\"117\" \/><\/p>\n<h3>Example#6: Using \u2013D option<\/h3>\n<p>The following command will print all duplicate lines from the file.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/linuxhint.com\/wp-content\/uploads\/2018\/12\/6-32.png\" alt=\"\" width=\"733\" height=\"135\" \/><\/p>\n<h3>Example#7: Using \u2013all-repeated option with prepend method<\/h3>\n<p>Three methods can be used with \u2013all-repeated option which are mentioned earlier of this tutorial. Here, the prepend method is used with this option that prints duplicate lines by appending blank lines at the beginning of duplicate lines.<\/p>\n<p>$ uniq &#8211;all-repeated=prepend uniq_test.txt<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/linuxhint.com\/wp-content\/uploads\/2018\/12\/7-33.png\" alt=\"\" width=\"733\" height=\"148\" \/><\/p>\n<h3>Example#8: Using \u2013u option<\/h3>\n<p>The following command will find out all the unique lines from the file. There are three unique lines in uniq_test.txt file which are printed as output.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/linuxhint.com\/wp-content\/uploads\/2018\/12\/8-30.png\" alt=\"\" width=\"736\" height=\"153\" \/><\/p>\n<h4>Conclusion<\/h4>\n<p>The uses of uniq command are explained and shown by using various examples in this tutorial. Hope, you will be able to use uniq command properly after reading this tutorial.<\/p>\n<p><a href=\"https:\/\/linuxhint.com\/bash_uniq_command\/\" target=\"_blank\" rel=\"noopener\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Linux users need to create or read the text file in regular basis for many purposes. A text file can contain different types of numeric and character data. Same data can be stored multiple times in a text file. Sometimes, you may require reading any text file by omitting duplicate lines of data. Bash uniq &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.appservgrid.com\/paw92\/index.php\/2018\/12\/29\/bash-uniq-command-linux-hint\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Bash uniq Command \u2013 Linux Hint&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-6925","post","type-post","status-publish","format-standard","hentry","category-linux"],"_links":{"self":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/6925","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/comments?post=6925"}],"version-history":[{"count":1,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/6925\/revisions"}],"predecessor-version":[{"id":6976,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/posts\/6925\/revisions\/6976"}],"wp:attachment":[{"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/media?parent=6925"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/categories?post=6925"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.appservgrid.com\/paw92\/index.php\/wp-json\/wp\/v2\/tags?post=6925"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}