App::highlight - a simple grep-like highlighter app
Upcoming SlideShare
Loading in...5
×
 

App::highlight - a simple grep-like highlighter app

on

  • 8,836 views

App::highlight is a bit like grep, except that it doesn't filter out lines. In exchange for seeing all the output you get a lot more fun highlighting options to play with, and full Perl regex support ...

App::highlight is a bit like grep, except that it doesn't filter out lines. In exchange for seeing all the output you get a lot more fun highlighting options to play with, and full Perl regex support of course.

I gave this talk at the London.pm technical meeting in July 2013.

App::highlight is available on Github and CPAN.

Statistics

Views

Total Views
8,836
Views on SlideShare
6,284
Embed Views
2,552

Actions

Likes
5
Downloads
6
Comments
4

17 Embeds 2,552

https://bozuman.cybozu.com 2003
http://blogs.perl.org 182
https://bozuman.s.cybozu.com 170
http://cloud.feedly.com 85
http://backend-toreader.rhcloud.com 30
https://twitter.com 26
http://news.int80.biz 15
http://www.feedspot.com 13
http://toreader-toreader.rhcloud.com 7
http://summary 6
http://digg.com 5
http://newsblur.com 3
http://tweetedtimes.com 3
http://www.newsblur.com 1
http://www.goread.io 1
http://localhost 1
http://toreadr-tunes.rhcloud.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • @MichaelJemmeson Sure, no problem :) Hope to see you at the next London.pm tech meeting!
    Are you sure you want to
    Your message goes here
    Processing…
  • @kaokun thanks for the reply. I couldn't make the talks in the end and didn't see it mentioned in the slides or the module so just thought I'd mention it
    Are you sure you want to
    Your message goes here
    Processing…
  • @MichaelJemmeson true, I even mentioned that when I was giving the talk :) The first feature App::highlight got that (afaik) grep doesn't have was the multiple colours that it cycles through for each different match. Very handy when you're looking for more than one thing or you're using it in tricks like the svndiff alias.
    Are you sure you want to
    Your message goes here
    Processing…
  • It looks like you have more functionality, but FYI you can do highlighting of the complete file with grep, although it's a side effect really: grep -h --color -E 'foo|$' (that's pipe then dollar)
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

App::highlight - a simple grep-like highlighter app App::highlight - a simple grep-like highlighter app Presentation Transcript

  • App::highlight a simple grep-like highlighter app Alex Balhatchet @ London.pm Technical Meeting, July 2013
  • Intro alex@masaki:~$ cat words.txt fred barney betty wilma dino pebbles bamm-bamm
  • Intro alex@masaki:~$ cat words.txt | grep a barney wilma bamm-bamm
  • Intro alex@masaki:~$ cat words.txt | highlight a fred barney betty wilma dino pebbles bamm-bamm
  • Why?
  • Real World Example 1 alex@masaki:~$ servicectl status servicectl status: checking arbyte-job-buffer-probe... running. servicectl status: checking arbyte-job-runner-cleanup-cron... running. servicectl status: checking arbyte-job-runners... running. servicectl status: checking arbyte-manager... running. servicectl status: checking arbyte-status-accepter... running. servicectl status: checking imagesys-managerd... running. servicectl status: checking launch-target-cron... running. servicectl status: checking listings-db... running. servicectl status: checking ops-cron... not running. servicectl status: checking search-db... running. servicectl status: checking searchindex-cron... running. servicectl status: checking updatesem-cron... running.
  • Real World Example 1 alex@masaki:~$ servicectl status | highlight -l 'not running' servicectl status: checking arbyte-job-buffer-probe... running. servicectl status: checking arbyte-job-runner-cleanup-cron... running. servicectl status: checking arbyte-job-runners... running. servicectl status: checking arbyte-manager... running. servicectl status: checking arbyte-status-accepter... running. servicectl status: checking imagesys-managerd... running. servicectl status: checking launch-target-cron... running. servicectl status: checking listings-db... running. servicectl status: checking ops-cron... not running. servicectl status: checking search-db... running. servicectl status: checking searchindex-cron... running. servicectl status: checking updatesem-cron... running.
  • Real World Example 2 alex@masaki:~$ zcat listings.txt.gz | grep "'id' => '8244118'" $VAR1 = {'debug' => {'remote_used' => 1},'etl' => {'agent_database_key' => undef,'feed_version' => '1.7','feed_timestamp' => '20130717100846','id' => '8244118','feed_segment' => undef,'feed_name' => 'domain','partner_database_key' => undef},'listing_attributes' => {'agent_description' => undef,'gross_size' => undef,'floor' => '1','num_baths' => 1,'gross_size_unit' => undef,'construction_year' => undef,'num_rooms' => undef,'num_beds' => 3,'commission' => undef,'property_type' => 'house','datasource_custom_two' => undef,'land_size' => undef,'datasource_custom_five' => undef,'net_size_unit' => undef,'energyrating' => '0','net_size' => undef,'agent_logo_url' => undef,'rental_deposit' => undef,'energyindex' => '0','car_spaces' => '1','land_size_unit' => undef,'datasource_custom_three' => undef,'auction_date' => undef,'datasource_custom_four' => undef,'datasource_custom_one' => undef},'listers' => {'name' => 'Private Advertiser','type' => undef,'company' => 'Private Advertiser'},'listings' => {'price_currency' => 'AUD','full_description' => 'Alarm System, Balcony, Garden, Internal Laundry, Polished Timber Floors, Built-In Wardrobes, Fireplaces, Renovated - 6 metres wide, grand and spacious, light- filled terrace Restoration by award-winning architect 5 minute stroll to Woollahra village, Centennial Park and Oxford Street Enormous family/dining/kitchen area opening to rear courtyard garden Double Living area with marble fireplaces and mirrors 3 large bedrooms - two with marble fireplaces, all with quality built-ins and plantation shutters Rear lane car access via remote control roll-a-door Luxury marble bathroom with bath and separate frameless glass shower Built-in laundry Downstairs powder room Large, private rear courtyard with established garden Entrance courtyard garden Garden maintenance included Foxtel connections to living areas and two bedrooms Alarm system Telephone 0412 758 772 to arrange an appointment for inspection','placenames' => '{"street":"Windsor Street","building":"157","province":"NSW","town":" Paddington"}','listing_type' => 'rent','title_keywords' => 'paddington,windsor street','keywords' => ['backyard','balcony','garden','kitchen','terrace'],'actually_the_original_address' => '157 Windsor Street, Paddington, NSW, 2021','latitude' => '-33.886527','display_address' => undef,'raw_content_digest' => 'c9080b2827fbf612aec4369dd6af1931','debug' => undef,'address' => '157 Windsor Street, Paddington, NSW, 2021','price_type' => 'weekly','longitude' => '151.235183','teaser' => 'Alarm System, Balcony, Garden, Internal Laundry, Polished Timber Floors, Built-In Wardrobes, Fireplaces, Renovated 6 metres wide, grand and light-filled terrace Restoration by award-winning architect 5 minute stroll to Woollahra village, Centennial Park and Oxford Street Enormous...','geocode_method' => 'remote','geocode_accuracy' => 9,'photo_url' => 'http://images.domain.com.au/img/2013320/0/158ffe47-3469-466b-9727-a51c4def7e1b_FS.jpg','datasource_dbkey' => '8244118','site_url' => 'http://www.domain.com.au/Public/PropertyDetails.aspx? adid=8244118','photo_count' => '8','ext_categories' => '','price' => 1850,'coldrent' => undef,'title' => 'Windsor Street, Paddington - Balcony','datasource_name' => 'domain','price_low' => 1850,'photos' => {'urls' => ['http://images.domain.com.au/img/2013320/0/158ffe47-3469-466b-9727-a51c4def7e1b_FS.jpg'],'total' => '8'},'monetized' => '0','status_type' => 'active','datasource_campaign_id' => undef,'site_mobile_url' => undef,'display_brand' => undef,'attribute_hash' => '8244118','postcode' => '2021','country' => undef,'orig_address' => '157 Windsor Street, Paddington, NSW, 2021','geocode_matched' => '2021, 157 Windsor Street, Paddington, New South Wales, Australia','price_high' => 1850,'display_price' => '$1,850'}};
  • Real World Example 2 alex@masaki:~$ zcat listings.txt.gz | grep "'id' => '8244118'" | highlight 'postcode' $VAR1 = {'debug' => {'remote_used' => 1},'etl' => {'agent_database_key' => undef,'feed_version' => '1.7','feed_timestamp' => '20130717100846','id' => '8244118','feed_segment' => undef,'feed_name' => 'domain','partner_database_key' => undef},'listing_attributes' => {'agent_description' => undef,'gross_size' => undef,'floor' => '1','num_baths' => 1,'gross_size_unit' => undef,'construction_year' => undef,'num_rooms' => undef,'num_beds' => 3,'commission' => undef,'property_type' => 'house','datasource_custom_two' => undef,'land_size' => undef,'datasource_custom_five' => undef,'net_size_unit' => undef,'energyrating' => '0','net_size' => undef,'agent_logo_url' => undef,'rental_deposit' => undef,'energyindex' => '0','car_spaces' => '1','land_size_unit' => undef,'datasource_custom_three' => undef,'auction_date' => undef,'datasource_custom_four' => undef,'datasource_custom_one' => undef},'listers' => {'name' => 'Private Advertiser','type' => undef,'company' => 'Private Advertiser'},'listings' => {'price_currency' => 'AUD','full_description' => 'Alarm System, Balcony, Garden, Internal Laundry, Polished Timber Floors, Built-In Wardrobes, Fireplaces, Renovated - 6 metres wide, grand and spacious, light- filled terrace Restoration by award-winning architect 5 minute stroll to Woollahra village, Centennial Park and Oxford Street Enormous family/dining/kitchen area opening to rear courtyard garden Double Living area with marble fireplaces and mirrors 3 large bedrooms - two with marble fireplaces, all with quality built-ins and plantation shutters Rear lane car access via remote control roll-a-door Luxury marble bathroom with bath and separate frameless glass shower Built-in laundry Downstairs powder room Large, private rear courtyard with established garden Entrance courtyard garden Garden maintenance included Foxtel connections to living areas and two bedrooms Alarm system Telephone 0412 758 772 to arrange an appointment for inspection','placenames' => '{"street":"Windsor Street","building":"157","province":"NSW","town":" Paddington"}','listing_type' => 'rent','title_keywords' => 'paddington,windsor street','keywords' => ['backyard','balcony','garden','kitchen','terrace'],'actually_the_original_address' => '157 Windsor Street, Paddington, NSW, 2021','latitude' => '-33.886527','display_address' => undef,'raw_content_digest' => 'c9080b2827fbf612aec4369dd6af1931','debug' => undef,'address' => '157 Windsor Street, Paddington, NSW, 2021','price_type' => 'weekly','longitude' => '151.235183','teaser' => 'Alarm System, Balcony, Garden, Internal Laundry, Polished Timber Floors, Built-In Wardrobes, Fireplaces, Renovated 6 metres wide, grand and light-filled terrace Restoration by award-winning architect 5 minute stroll to Woollahra village, Centennial Park and Oxford Street Enormous...','geocode_method' => 'remote','geocode_accuracy' => 9,'photo_url' => 'http://images.domain.com.au/img/2013320/0/158ffe47-3469-466b-9727-a51c4def7e1b_FS.jpg','datasource_dbkey' => '8244118','site_url' => 'http://www.domain.com.au/Public/PropertyDetails.aspx? adid=8244118','photo_count' => '8','ext_categories' => '','price' => 1850,'coldrent' => undef,'title' => 'Windsor Street, Paddington - Balcony','datasource_name' => 'domain','price_low' => 1850,'photos' => {'urls' => ['http://images.domain.com.au/img/2013320/0/158ffe47-3469-466b-9727-a51c4def7e1b_FS.jpg'],'total' => '8'},'monetized' => '0','status_type' => 'active','datasource_campaign_id' => undef,'site_mobile_url' => undef,'display_brand' => undef,'attribute_hash' => '8244118','postcode' => '2021','country' => undef,'orig_address' => '157 Windsor Street, Paddington, NSW, 2021','geocode_matched' => '2021, 157 Windsor Street, Paddington, New South Wales, Australia','price_high' => 1850,'display_price' => '$1,850'}};
  • Features
  • Features alex@masaki:~$ highlight --help highlight [-bCcehlnor] [long options...] -c --color use terminal color for highlighting -C --no-color don't use terminal color -e --escape auto-escape input (default) -r -n --regex --no-escape don't auto-escape input (regex mode) -i --ignore-case ignore case for matches -l --full-line highlight the whole matched line -o --one-color use only one color for all matches -b --show-bad-spaces highlight spaces at the end of lines -v --version show version number -h --help display a usage message
  • Color alex@masaki:~$ cat words.txt | highlight fred barney betty wilma dino pebbles bamm- bamm fred barney betty wilma dino pebbles bamm-bamm
  • No Color alex@masaki:~$ cat words.txt | highlight fred barney betty wilma dino pebbles bamm- bamm --no-color <<fred>> [[barney]] ((betty)) {{wilma}} **dino** __pebbles__ <<bamm-bamm>>
  • Escape alex@masaki:~$ cat words.txt | highlight 'b[ea]t{2}.' fred barney betty wilma dino pebbles bamm-bamm
  • No Escape (aka regex mode) alex@masaki:~$ cat words.txt | highlight --regex 'b[ea]t{2}.' fred barney betty wilma dino pebbles bamm-bamm
  • Ignore Case alex@masaki:~$ cat words.txt | highlight --ignore-case 'MM' fred barney betty wilma dino pebbles bamm-bamm
  • Full Line alex@masaki:~$ cat words.txt | highlight --full-line 'a' fred barney betty wilma dino pebbles bamm-bamm
  • One Color alex@masaki:~$ cat words.txt | highlight --one-color 'f' 'r' 'e' 'b' fred barney betty wilma dino pebbles bamm-bamm
  • Bad Spaces (inspired by git diff) alex@masaki:~$ cat words.txt | highlight --show-bad-spaces fred barney bettyxxxx wilma dino pebblesx bamm-bamm
  • Cool Tricks
  • SVN Diff alias svndiff=" svn diff | highlight --show-bad-spaces | highlight --regex --full-line '^-[^-]' '^[+][^+]' '^[+-]{2}' | less -R"
  • SVN Diff alex@masaki:~$ svndiff =================================================================== --- SERP.pm (revision 62240) +++ SERP.pm (working copy) @@ -1,13 +1,14 @@ +use utf8; +xxxx package Lokku::URL::SERP; use base 'Lokku::URL'; -use Clone 'clone'; -use List::MoreUtils 'uniq'; +use Clone qw(clone); +use List::MoreUtils qw(uniq); use Lokku::Base::Config;
  • Tailing Access Logs alex@masaki:~$ tail -f ~/common/logs/apache/frontend/access_log.2013- 07-17-00_00_00 | highlight -l -r 'HTTP/1.1" 50d ' 'HTTP/1.1" 200 ' 'HTTP/1.1" 30d ' 'HTTP/1.1" 40d ' x.x.x.x - - [17/Jul/2013:14:13:51 +0000] "GET /abode-sale-house-in-friday-hill HTTP/1.1" 200 9549 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 125669 www x.x.x.x - - [17/Jul/2013:14:13:51 +0000] "GET /paige-petrook-rent-flat-in-the-avenue-ha5 HTTP/1.1" 200 9586 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 97016 www x.x.x.x - - [17/Jul/2013:14:13:51 +0000] "GET /gunness/property/buy HTTP/1.1" 200 16501 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 109951 www x.x.x.x - - [17/Jul/2013:14:13:52 +0000] "GET /rd?l=UbeTFSQlTSlUJUMFlk&url=2- ePz73aTBU4LVGFByDI484Ner8aIMqOimmnO-vZos3KDcnfaeeMPe4rGzMAHEMEMIx5MZsli8WOfUu6sPU8-oGyl- oBxFgBrDY1PNVVZlNerg11bkCYzKZwBlYrKuUnoqgEDyvQYT0AWjNiAXvEvzby0n8wKI_4SYr67NnK50C3WeiSB3ATzAdHRuPx89ZkLfV 7zVXG2QDbeS6DmC6o17lS3zg4TQbgURP9KPebQfsYvRw- KZZSJe5bz67v9wC0YOb6G85BbGUnJzoOHf6gadGwCRAqFSnkZv4Fqu84Vq_WqIfy_oPVJlWQ%3D% 3D&v=2&s=Vbelwusrlloqtlnpusmosplslplu&itype=1 HTTP/1.1" 302 - "http://www.nestoria.co.uk/victoria-road- ct14/house/sale" "Mozilla/5.0 (Linux; Android 4.1.2; GT-P3100 Build/JZO54K) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.64 Safari/537.36" 10465 rd x.x.x.x - - [17/Jul/2013:14:13:52 +0000] "GET /help/unknown-location HTTP/1.1" 404 9279 "-" "Mozilla/5.0 (Linux; U; en-gb; KFOT Build/IML74K) AppleWebKit/535.19 (KHTML, like Gecko) Silk/3.1 Safari/535.19 Silk- Accelerated=true" 5932 www x.x.x.x - - [17/Jul/2013:14:13:53 +0000] "GET /api? action=search_listings&encoding=json&sort=price_lowhigh&price_min=150001&price_max=200000&page=1&radius=5 2.360992,-2.039034,1mi HTTP/1.1" 200 23729 "-" "-" 65233 api
  • Using /usr/bin/watch with highlight alex@masaki:~$ watch --color 'du -hsc * | highlight -l G K M' 1.1G alex 32M davidl 16K savio 1.3G total
  • Tech Details
  • Perl Modules Used For the app App::Cmd::Simple Term::ANSIColor Getopt::Long::Descriptive For testing App::Cmd::Tester Test::Without::Module
  • Bug Reports / Contributions https://metacpan.org/module/App::highlight Version 0.13 released July 18th 2013 :-) https://github.com/kaoru/App-highlight https://travis-ci.org/kaoru/App-highlight
  • N estoria is H iring! http://lokku.com /jobs Thanks!