HowTo: Extract Lines That Started With Matching String Using Grep

From recent post of displaying the first occurrence of line that starts with a specified matching string, now instead of the first occurrence, here's a way of stripping out all matching lines that start with the given matching string.

Reproducable Approach
Say for example, we have a text file with the below contents and we need to extract those lines shown in BOLD.

# cat testfile.txt
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
my linux sysad blog
daily linux sysad blog
linux systems admin blog
systems admin blog
systems admin blog matching line 2
systems admin blog matching line 3
fedora systems admin blog matching line 4
linux systems admin blog matching line 5
pinoy systems admin blog matching line 6
blogging fedora sysad blog
fedora syad blog
distrowatch and linux distributions
linux systems admin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Noticed above that the words "systems admin blog" appeared a lot of times from different lines. Our objective is to extract ALL the occurring lines that exactly started with the given query string, which is systems admin blog .

Here's how to extract line occurrences that starts with the word "systems admin blog" words. Simply

# grep "^systems admin blog" testfile.txt

Alternatively, using linux pipe

# cat testfile.txt | grep "^systems admin blog"

output:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
systems admin blog
systems admin blog matching line 2
systems admin blog matching line 3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The above commands comes in useful when you need to extract lines that starts with the given query string from a thousand lines of text files. Simple but useful when managing large text file contents.

Take note that you'll get a different results if you do

# grep "systems admin blog"

That is all folks, enjoy.

HowTo: Delete All Thumbs.db Recursively

An offline messages just arrive from a fedora newbie trying to polish more of his FTP administration skills.

After quite some time of establishing his fully functional FTP server via VSFTP, he then realized that most of their staff with windows-based desktops usually Selects ALL files, Copy and Paste to FTP site, the normal non-techie way. Unaware that selecting all files from currently working windows folder also selects the Thumbs.db file to be uploaded to that same FTP server.

This Thumbs.db file is by the way created by windows OS for the purpose of thumbnailing cached image files, which has a filesize of 23040. If you've 1,000 of that, you just lost 23MB of disk space.

Now, the answer to his question of deleting Thumbs.db file recursively following the same concept with recent postings. This can be simply done using find.


# find ~ftpusername -name "Thumbs.db" -exec rm -rf {} \;


Adding to cron job on every morning or on daily basis would be


01 07 * * * find ~ftpusername -name "Thumbs.db" -exec rm -rf {} \; > /dev/null 2>&1


Goodluck, hope this helps.

Related External Articles:
What Happens After rm -rf / ?

HowTo: Delete All Thumbs.db Recursively

HowTo: Find And Count JPG Files Recursively

Delete Zero-sized Filename Recursively

Find Hidden Linux File Recursively

Move Linux Files Recursively

Rename Linux File Recursively

Delete Files and Folders Recursively

HowTo: Handy Linux Network Diagnostic Tool

I should have posted this handy linux command long time ago, which was also mentioned from this link, but not explicitly eloborated.

What is MTR?

Mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.

Mtr, when issued, investigates the network connection between the current host where mtr runs on and another host by sending packets with purposely low TTLs. Mtr continues to send packets with low TTL, noting the response time of the intervening routers. This allows mtr to print the response percentage and response times of the internet route to target host. A sudden increase in packetloss or response time is often an indication of bad, maxing out or simply overloaded internet link.

This mtr linux tool comes handy when you need to troubleshoot and isolate internet link between two hosts. Mtr can accept command line parameters as input command customization.

By default, mtr runs with reverse lookup feature enabled, resolving each trace hops to its equivalent hostname rather than IP addresses. Mtr is capable of executing traceroute and ping command both via IPv4 and IPv6, which is quite ready for future IP address worldwide evolution. Packet size and packet counts can also be specified explicitly, which increases command control over packets being sent to and from hosts. Mtr continues to send packets to and from providing statistical traceroute date between hosts including average travel time, latency, traceroute percentage and packet loss, worst and best packet replies and ping echoes.

Interactive commands are also available using mtr. Display mode can be toggled between textual visual display to numerical display, which provides continuous long running convenience over hosts being isolated for possible intermittent links and routes. Fields sorting is also being supported including the order of fields for further display customization.

Mtr Installation

Mtr comes as a single rpm package, which means, it is not part of any linux package and is available from Fedora package. Mtr is installed by default Fedora base installation. To install, simply

# yum -y install mtr

Binary Usage

# mtr google.com
# mtr ilovetux.com --no-dns
# mtr --raw gmail.com
# mtr --split googlepages.com

If you have issues with your BGP routing and multi source routes, mtr can be quite handy troubleshooting from linux box level for a more detailed routing isolation and troubleshooting when advanced routing seems so fuzzy.

All in all, mtr comes very handy, not as a replacement but as an alternative and continuous network diagnostic linux tool for ping and traceroute linux binary commands.

That is all, enjoy.

Related Articles:
Linux Magic Ping Command Explicitly Explained
Sending Ping To IP Subnet Blocks
Overly Used and Handy Linux Port Scanning Tool
Sending Ping to Nokia N70
Other Alternative and Interesting Ping Related Commands

HowTo: Display Lines That Starts With A Given String

Matching strings can be counted using grep, cat, sed. Matching strings can be sorted and even omitted for matching repetitions using sort and uniq. They can also be highlighted during the string parsing operation via grep. More can be done using awk. But have you ever tried to display the first occurrence of matching line that exactly begins or starts with the given search string?

Our objective is to display only the first line of occurrence that exactly begins with our matching string "blog linux" (shown in bold - 4th line), see below.

# cat testfile.txt
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
blogger and blogger and linux blog linux
blogging linux blog linux redhat blog linux
blogging sysad blogs fedora linux blog linux
blog linux followed by any words
blogs linux sysad blog linux and blog linux
blogs sysad blog fedora blog linux
fedora blog systems blog linux
linux blog linux blog linux
sysad blog linux blog linux
sysad blogs linux blog linux
systems admin blog linux sysad blog linux
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Noticed that the word "blog linux" appeared several times from different lines of that text file. What we only need is the first occurrence of line that started with the word "blog linux" as our matching string.

Here's how to display only those matching line from text file that exactly begins with given search string.

First, we need to sort the input text file like so

# cat testfile.txt | sort | uniq > output.txt

Then, let's look that matching line that begins with our given search string "blog linux". This would be done using linux look binary command.

# look -d "blog linux" output.txt
result:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
blog linux followed by any words
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

That is all, enjoy.

Random Articles:
Quick MRTG Install Tips
MRTG with SkyStream DVB Router and Satellite Receiver
Nagios Generic Setup and Install Tips
Disk Space Report using Shell
Monitoring Server Hosts via Munin

HowTo: Find And Count JPG Files Recursively

I have been busy playing around with more than 100,000 images and JPG files from my hard drive and I just realized that I was issuing a few set of commands redundantly specially on counting the number of images from categorical 'named' folders alone and from its sub 'event-folders' recursively.

Here's how to find and count JPG image files from subdirectories recursively.

Assuming a list of thousand JPG image files
# cd /images
# ls -la
result:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drwxr-xr-x 51 root root 4096 2008-02-25 18:10 image00001
..
snip
drwxr-xr-x 51 root root 4096 2008-02-25 18:10 image99999
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

From current working folder location, start finding and counting JPG files
# find . -name "*.jpg" | wc -l

By default, the command find searches JPG files recursively on current working directory. It should be able to find the supplied search filename.

Alternatively, creating an alias with the same command could also be convenient

# alias countjpg='find . -name "*.jpg" | wc -l'

Just remember to make it permanent between reboots by simply appending it to your ~/bashrc file.

Other prefers it to be coming from their own bash shell script

# cat /bin/countjpg.sh
output
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#!/bin/bash
clear
echo Recursively counting JPG files on current folder $(pwd) ...
find . -name "*.jpg" | wc -l
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# chmod 700 /bin/countjpg.sh
# countjpg.sh

That is all, enjoy.

Few Most Popular Articles:
Watch and Play 113 Amazing Linux Games?
Enabling Sun Java on Fedora 8
50 Quick Linux Tips Part 1

HowTo: Do Awk with Loop

From recent posts, we have tried to remove the first word by statically specifying the matching string fields using awk and cut.

What if we need to remove the first word but retain the 2nd word up to Nth word of a line from a text file?

Here's a real quick and easy of dynamically looping awk to achieve the above effects.
Now let us try removing the first word while retaining the 2nd word up to the Nth word of a line from any text files.

# cat testfile.txt
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Now we have removed the first word while retaining the rest from 2nd to Nth word of a line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# cat testfile.txt | awk '{for(i=2;i<=NF;i++) printf ("%s ", $i);}'
output
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
we have removed the first word while retaining the rest from 2nd to Nth word of a line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

That is all folks.

HowTo: Strip Out First Word of Line from Text Files

This is a quick post on how to remove the first word of a line.

There are lot of ways on how to remove and strip out the first word of a line from a text file. Here's one way to achieve that.

# cat testfile.txt
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
my blog name is sysad linux blog
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Assuming we need to strip down or remove the word 'my' from the above line, that would be basically easy using linux binary tools. One approach would be using awk like so

Here's how to remove the first word of a line from textfiles using static approach via awk and cut linux tool.

Working on same content of previous file
# cat testfile.txt | awk '{print $2,$3,$4,$5,$6,$7}'
output
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
blog name is sysad linux blog
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Using cut, that would be

# cat testfile.txt | cut -d' ' -f2,3,4,5,6,7

That's it, enjoy.

Related Articles:
Get the Filename Extension Only
Strip Out Directory Path and Filename Suffixes
Removing New Lines and Return Lines from Text Files
Have you ever head of TR in Linux?
Inverting or Reversing Grep Matches
Strip Out Non-Directory Path from Filename
Removing Spaces from Filenames
Removing Blank Lines using Sed Or Grep
50 Quick Linux Command Tips Part 4

HowTo: Highlight Matching Grep Results

Here's a quick entry on how to have a highlighted and colored text from grep resulting strings and characters.

Assuming we have /var/log/maillog and we wish to grep it with multiple search strings. Well this would be faily simply

# cat /var/log/secure | grep 'Accept\|refuse'

What if we want to make resulting matching grep strings appear in colored text or highlighted text. Highlighted and colored characters or words appear much more highlighted and distinguished specially when you are viewing large number of text lines from a log file.

Here's how to set and kick the ball.

Say from root terminal, simply issue

# alias grep='grep --color=auto'

And try issuing the later command again like so

# cat /var/log/secure | grep 'Accept\|refuse'

Make permanent between reboots for user root.

# echo "alias grep='grep --color=auto'" >> /root/.bashrc

Could be useful on tailing and grepping log files when needed.

Quick and easy. Enjoy.

HowTo: GNOME Commander File Manager

GNOME Commander is a nice and fast file manager for the GNOME desktop. In addition to performing the basic file manager functions, this feature-riched file manager program is also a FTP-client software at the same time. GNOME Commander can also browse shared SMB-networks or SMB-drive.

GNOME Commander is very similar to text-based file manager called midnight commander. GNOME Commander aims to fulfill the demands of more advanced users who like to focus on file management, their work through special applications and running smart commands inside their X display. GNOME Commander can display two-pane file management windows which can is available and controllable on demand and when needed. Shortcut keyboard keys makes file management and operation with your desktop much easier than before.

GNOME Commander Highlight Features:

* GTK-2 GUI with standard mouse interactions.
* Gnome mime types (MIME-Type management under development).
* FTP using GnomeVFS ftp module.
* SAMBA access.
* Right click mouse menu with the usual file operations to run, open, open with.., delete or rename files and folders, and also to set properties like ownership and permissions.
* Mouse context menu that easily can be extended by entries calling any kind of external application like viewers, editors or custom scripts to work on selected files or directories.
* Quick device access buttons with automatic mounting and unmounting ("no-mount" option for just quick access to folders or otherwise automounted devices).
* Latest accessed folder history.
* Folder bookmarks.
* Plugin support under development.
* Fast file viewer for text, images and image meta data (Exif and IPTC)..
* Tools for searching, quick file name search in current dir, symlinking, comparing directories. ...
* A tool for advanced renaming of files supporting various types of meta-data.
* Integrated command line.

GNOME Commander Fedora Installation
# yum -y install gnome-commander

GNOME Commander Binary

# gnome-commander

GNOME Commander File Manager Webshot



That is all, enjoy.

Related Articles:
Thunar File Manager
Linux Profile Manager
Krusader - Two-Pane File Manager

HowTo: Install Plone CMS on Fedora

From recent Wordpress installation on Fedora system, here's another quick installation howto of Plone - a content management system (CMS) on Fedora system

What is Plone?

Plone is a user friendly and powerful content management system based on Zope. Plone is easy to use, translated into over 35 languages and carefully follows standards. It's under active development and easily extensible with add-on products. Sites powered by plone.

Plone Installation on Fedora

Plone package is available from livna repository. To install livna on Fedora 7, go here, installation on Fedora 8 should be likely similar.

To install Plone CMS on Fedora 8
# yum -y install compat-plone

The above command would download around 13MB of rpm package files.

Learn more of Plone, it's one of nice content management system around.

Similar Article:
WordPress Installation on Fedora 8

HowTo: Thumbnail A Website From Linux Command Terminal

Blogging from Wordpress platform enables a blogger to incorporate and activate snapshot plugin where a blogger create on-demand URL snap shots of their websites.

If you are looking for HTML site thumbnailer software available from Linux command lin, read on.

Here's how to create a thumbnail images of URL sites from linux terminal.

Gnome Web Photo contains a thumbnailer that will be used by GNOME applications, including the file manager, to generate screenshots of web pages.

Gnome-Web-Photo Thumbnailer Fedora Installation

To generate thumbnail images using gnome-web-photo
# yum -y install gnome-web-photo

Sample Usage:

Creating thumbnails image of a website or URL site.
# gnome-web-photo -t 0 --mode=thumbnail -w 1152 -s 128 http://www.yahoo.com yahoo.jpg
# gnome-web-photo -t 0 --mode=thumbnail -w 1152 -s 256 http://www.yahoo.com yahoo2.jpg
# gnome-web-photo -t 0 --mode=thumbnail -w 512 -s 256 http://www.google.com google.jpg

After a while, try to view the thumbnail images
# gthumb google.jpg
# gview yahoo.jpg

Gnome-web-photo also supports create snapshot in photo mode which creates a wider and larger version of URL site in photo mode. This feature can also be applied to actual image files from your hard drive, giving you an alternative way to scale down your photo images like so

# gnome-web-photo -w 64 --files /should/have/full/path/food.jpg thumb.jpg

Creating thumbnails images of URL in photo mode for larger image and better quality thumbnail pictures
# gnome-web-photo -t 0 --mode=photo -w 1152 http://www.yahoo.com yehey.jpg ; gthumb yehey.jpg

Sample Created Thumbnail URL Images


That is all, enjoy.

Related Articles:
Photo-Content Management Gallery and System
Thumbnail Apache Linux Module Installation
KDE Photo Library Management
ImageMagic Fedora Installation
Google Picasa Installation HowTo
JPG to MPG File Conversion with Fedora

HowTo: Advanced Two-Pane File Manager

Have you read the last post of having a two-pane GUI file manager from your desktop linux? Well, here's another round of a much more advanced two-pane file manager.

What is Krusader

Krusader is the result of a 7-year old programming and discussion colloboration of opensource developer to come up with a unique and advance X file manager.

Krusader is an advanced twin panel (commander style) file manager for KDE and other desktops in the *nix world, similar to Midnight or Total Commander. Krusader provides all the file management features you could possibly want. Krusader provides other rich features such as extensive archive handling, mounted filesystem support, FTP, advanced search module, an internal viewer/editor, directory synchronisation, file content comparisons, powerful batch renaming and much much more.

Krusader supports a wide variety of archive formats and can handle other KIO slaves such as smb or fish. Krusader is almost completely customizable, very user friendly, fast and looks great on your desktop!

Krusader Feature Highlights
Highlights:

* Full support for ACL permissions like properties, preserve attributes, synchronizer and other
* Many terminal emulator enhancements bringing new functions, new usages and new look
* Many improvements of the UserAction system
* Countless usability enhancements, especially in Konfigurator and the heavily reordered menubar
* Comparing in the synchronizer interface and parallel threading over slower servers
* Atomic extensions and ability to rename the filename, without extension
* Numerous archive enhancements like handling packing in the background, encryption support, multiple volume archives, compress level


Krusader Installation on Fedora
# yum -y install krusader

Krusader Binary Launch
# krusader

Krusader comes in very handy specially when you're dealing with a lot of directory folders and files. You should give it a shot.

That is all, enjoy.

Some Webshots:

HowTo: Two-Pane GUI File Manager

Implementing a two-pane with GNOME File manager is still under review as an additional nautilus file manager feature. Having a two-pane file manager offers better flexibility, convenience and usability of file management and directory operations, which is unbeatable by any customized shell scripts.

Imagine the days when you have to view all the files and directory folders under the DOS command line environment. Technology of having a GUI-based file manager greatly eases out pains on managing files and folders from our hard drives. Now, imagine a GUI-based file manager with this two-pane X file manager.

Here's how to have a two-pane file manager from your X.

What is emelFM2?

emelFM2 is a file manager for UNIX-like operating systems. emelFM2 uses a simple and efficient interface pioneered by Norton Commander, in the 1980s. The main window is divided into three parts, described as "panes" or "panels". Two of those (side-by-side or top-to-bottom) show the contents of selected filesystem directories. The third pane, at the bottom of the window, shows the output of commands executed within the program. Those panes can be resized, and any one or two of them can be hidden and unhidden, on request. A built-in command-line, toolbar buttons or assigned keys can be used to initiate commands.

emelFM2 is the GTK+2 port of emelFM. emelFM2 is a file manager that implements the popular two-pane design. It features a simple GTK+2 interface, a flexible file typing scheme, and a built-in command line for executing commands without opening an xterm.

The usual file operation is commonly featured such as file deletions, directory folder creation, moving and renaming files, view and edit directory and more!

emelFM2 Fedora Installation
# yum -y emelfm2

Binary Launch
# emelfm2

Try it, it's fun, easy and great having a two-pane file manager around.

Sample Webshots:

Slash Proc - File System Utilities

One of the commonly used linux terminal commands are mostly the linux system and monitoring linux tools. Everybody who has been with linux for quite long time, knows that there are lots of built-in linux monitoring tools that comes free and handy from default linux OS installation.

One package that comes with a bunch of linux monitoring commands is procps package.

procps is the package that has a bunch of small useful utilities that give information about processes using, that is why it being referred to forward slash proc filesystem, the /proc filesystem.

The procps package contains a set of system utilities that provide system information. Procps includes ps, free, skill, pkill, pgrep, snice, tload, top, uptime, vmstat, w, watch and pdwx.

The ps command displays a snapshot of running processes.

The top command provides a repetitive update of the statuses of running processes.

The free command displays the amounts of free and used memory on your system.

The skill command sends a terminate command (or another specified signal) to a specified set of processes.

The snice command is used to change the scheduling priority of specified processes.

The tload command prints a graph of the current system load average to a specified tty.

The uptime command displays the current time, how long the system has been running, how many users are logged on, and system load averages for the past one, five, and fifteen minutes.

The w command displays a list of the users who are currently logged on and what they are running.

The watch program watches a running program.

The vmstat command displays virtual memory statistics about processes, memory, paging, block I/O, traps, and CPU activity.

The pwdx command reports the current working directory of a process or processes.

By default Fedora installation, procps is being installed.

Procps Installation

# yum -y install procps

That is all folks.

Related Readings:

Basic Proactive Linux Monitoring
Monitoring USB Devices
Proactive Log File Growth Monitoring
Being Aware of Logged On Users
Linux Process Kill
Linux Process Prioritization and Management
Linux Statistics and Resource Tool
Linux FileSystem Status

HowTo: Drop Down Linux Terminal Console

Are you looking for alternative linux terminal console that offers quite new terminal convenient features?

Here's an alternative linux tool that is available is also available from Fedora repos.

Yakuake - Yet Another Linux Terminal Console

Yakuake, one of the alternative KDE-based linux tool, is a drop-down terminal emulator based on KDE Konsole technology. Yakuake runs in background and pops out as dropdown window when called from shortcut keys. Yakuake also utilizes small memory footprints and therefor provide more memory resources to other linux applications. Yakuake customization is being offered from single point,click and release mouse gesture which makes it more convenient and easier to use. Toogle from one shell to another shell child is as easy as pressing key combinations, which comes as one of the application's default settings.

Yakuake also supports theme and skins. Window transparency is also available from this small but powerful linux terminal console. You can change from one color to another background color customization by simply selecting one of the default color schemes that comes from Yakuake linux terminal console.

Yakuake Key Features:

* Smoothly rolls down from the top of your screen
* Tabbed interface
* Configurable dimensions and animation speed
* Skinnable
* Sophisticated DCOP interface

Yakuake Installation
Simply install Yakuake via yum, which downloads about 82K of rpm package installer.

# yum -y install yakuake

Binary Launch

# yakuake

Some keyboard shortcuts that comes with Yakuake default configuration

Shift+Arrow - jumps from available opened linux terminal console
F12 - drops down terminal console

That is all, enjoy.

HowTo: Image Magics with ImageMagick

ImageMagick(TM) is an image display and manipulation tool for the X Window System. ImageMagick can read and write JPEG, TIFF, PNM, GIF, and Photo CD image formats. It can resize, rotate, sharpen, color reduce, or add special effects to an image, and when finished you can either save the completed work in the original format or a different one. ImageMagick also includes command line programs for creating animated or transparent .gifs, creating composite images, creating
thumbnail images, and more.

ImageMagick is one of your choices if you need a program to manipulate and dis play images. If you want to develop your own applications which use ImageMagick code or APIs, you need to install ImageMagick-devel as well.

There are times that when installing a particular image application, imagemagic is being searched for as part of its dependencies.

Here are steps of Installing ImageMagick on Fedora 8

ImageMagick Installation

# yum -y install ImageMagick

Says it all,