Mounting USB storage drives in Linux

Plug in Memory Stick into available USB port and then..

# dmesg | less

(The device is picked up as a USB 1.1 and allocated an address.
Also says what HCD it is using.)
usb 1-1: new full speed USB device using uhci_hcd and address 2

(SCSI emulation automatically kicks in)
scsi0 : SCSI emulation for USB Mass Storage devices
usb-storage: device found at 2

(Now the device information including model number is retrieved)
usb-storage: waiting for device to settle before scanning
Vendor: JetFlash  Model: TS512MJF2A        Rev: 1.00
Type:   Direct-Access                      ANSI SCSI revision: 02
SCSI device sda: 1003600 512-byte hdwr sectors (514 MB)

(The write-protect sense is EXPERIMENTAL code in the later kernels)
sda: Write Protect is off
sda: Mode Sense: 0b 00 00 08
sda: assuming drive cache: write through
SCSI device sda: 1003600 512-byte hdwr sectors (514 MB)
/dev/scsi/host0/bus0/target0/lun0: p1
Attached scsi removable disk sda at scsi0, channel 0, id 0, lun 0
Attached scsi generic sg0 at scsi0, channel 0, id 0, lun 0,  type 0
usb-storage: device scan complete
(At this point, the device is generally accessible by mounting /dev/sda1)

(When the device is disconnected, the system acknowledges the same)
usb 1-1: USB disconnect, address 2

To see the partition table type fdisk -l /dev/sda

To Mount the partition type

# mount /dev/sda1 /mnt/usb
# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda8             9.4G  7.5G  1.9G  80% /
/dev/hda9              11G  8.1G  2.4G  78% /usr
none                  189M     0  189M   0% /dev/shm
/dev/sda1             490M   34M  457M   7% /mnt/usb

Raid

RAID (redundant array of independent disks; originally redundant array of inexpensive disks) is a way of storing the same data in different places (thus, redundantly) on multiple hard disks. By placing data on multiple disks, I/O (input/output) operations can overlap in a balanced way, improving performance. Since multiple disks increases the mean time between failures (MTBF), storing data redundantly also increases fault tolerance.
A RAID appears to the operating system to be a single logical hard disk. RAID employs the technique of disk striping, which involves partitioning each drive's storage space into units ranging from a sector (512 bytes) up to several megabytes. The stripes of all the disks are interleaved and addressed in order.
In a single-user system where large records, such as medical or other scientific images, are stored, the stripes are typically set up to be small (perhaps 512 bytes) so that a single record spans all disks and can be accessed quickly by reading all disks at the same time.
In a multi-user system, better performance requires establishing a stripe wide enough to hold the typical or maximum size record. This allows overlapped disk I/O across drives.
There are at least nine types of RAID plus a non-redundant array (RAID-0):
• RAID-0: This technique has striping but no redundancy of data. It offers the best performance but no fault-tolerance.
• RAID-1: This type is also known as disk mirroring and consists of at least two drives that duplicate the storage of data. There is no striping. Read performance is improved since either disk can be read at the same time. Write performance is the same as for single disk storage. RAID-1 provides the best performance and the best fault-tolerance in a multi-user system.
• RAID-2: This type uses striping across disks with some disks storing error checking and correcting (ECC) information. It has no advantage over RAID-3.
• RAID-3: This type uses striping and dedicates one drive to storing parity information. The embedded error checking (ECC) information is used to detect errors. Data recovery is accomplished by calculating the exclusive OR (XOR) of the information recorded on the other drives. Since an I/O operation addresses all drives at the same time, RAID-3 cannot overlap I/O. For this reason, RAID-3 is best for single-user systems with long record applications.
• RAID-4: This type uses large stripes, which means you can read records from any single drive. This allows you to take advantage of overlapped I/O for read operations. Since all write operations have to update the parity drive, no I/O overlapping is possible. RAID-4 offers no advantage over RAID-5.
• RAID-5: This type includes a rotating parity array, thus addressing the write limitation in RAID-4. Thus, all read and write operations can be overlapped. RAID-5 stores parity information but not redundant data (but parity information can be used to reconstruct data). RAID-5 requires at least three and usually five disks for the array. It's best for multi-user systems in which performance is not critical or which do few write operations.
• RAID-6: This type is similar to RAID-5 but includes a second parity scheme that is distributed across different drives and thus offers extremely high fault- and drive-failure tolerance.
• RAID-7: This type includes a real-time embedded operating system as a controller, caching via a high-speed bus, and other characteristics of a stand-alone computer. One vendor offers this system.
• RAID-10: Combining RAID-0 and RAID-1 is often referred to as RAID-10, which offers higher performance than RAID-1 but at much higher cost. There are two subtypes: In RAID-0+1, data is organized as stripes across multiple disks, and then the striped disk sets are mirrored. In RAID-1+0, the data is mirrored and the mirrors are striped.
• RAID-50 (or RAID-5+0): This type consists of a series of RAID-5 groups and striped in RAID-0 fashion to improve RAID-5 performance without reducing data protection.
• RAID-53 (or RAID-5+3): This type uses striping (in RAID-0 style) for RAID-3's virtual disk blocks. This offers higher performance than RAID-3 but at much higher cost.
• RAID-S (also known as Parity RAID): This is an alternate, proprietary method for striped parity RAID from EMC Symmetrix that is no longer in use on current equipment. It appears to be similar to RAID-5 with some performance enhancements as well as the enhancements that come from having a high-speed disk cache on the disk array.

LPI Linux Certification in a Nutshell

You may not have heard of the Linux Professional Institute (LPI) or its professional certifications, but they're becoming an important part of proving professional competence in the Linux operating system. That aside, LPI Linux Certification in a Nutshell is a fantastic introductory Linux book, well suited to introducing a curious newcomer to the environment and bringing an intermediate user up to expert status.

The book is organized around the LPI's published standards for two Level 1 exams (exams 101, which deals with key commands and file-system concepts, and 102, which places more emphasis on hardware, networking, and shell scripting). The organization works well even if you're not specifically preparing for either exam.

LPI Linux Certification in a Nutshell assumes nothing in early chapters, going so far--to cite one example--as to walk readers through the concept of commands with parameters separately from the concept of commands alone. Later, the pace picks up, and strategic advice is substituted (such as how to partition a disk for maximum speed and reliability) for "type-this" instructions.

Throughout, the book makes effective use of O'Reilly's time-tested and remarkably clear format for presenting Unix commands and configuration files. Each chapter concludes with a series of exercises designed to help you discover behaviors on your own, and includes the practice questions you expect in a test-prep aid. --David Wall

Topics covered: The knowledge that's tested on the Linux Professional Institute's exams 101 and 102, which includes everything from basic Linux commands and concepts to installation of the operating system, essential network configuration, and kernel recompilation. --This text refers to the Paperback edition.

Book Description

LPI Linux Certification in a Nutshell, Second Edition is an invaluable resource for determining what you need to practice to pass the Linux Professional Institute exams. This book will help you determine when you're ready to take the exams, which are technically challenging and designed to reflect the skills that administrators need in real working environments.

As more corporations adopt Linux as the networking backbone for their IT systems, the demand for certified technicians will become even greater. Passing the LPI exams will broaden your career options because the LPIC is the most widely known and respected Linux certification program in the world. Linux Journal recognized the LPI as the best Training and Certification Program. The exams were developed by the Linux Professional Institute, an international, volunteer-driven organization with affiliates in a dozen countries.

The core LPI exams cover two levels. Level 1 tests a basic knowledge of Linux installation, configuration, and command-line skills. Level 2 goes into much more depth regarding system troubleshooting and network services such as email and the Web. The second edition of LPI Linux Certification in a Nutshell is a thoroughly researched reference to these exams. The book is divided into four parts, one for each of the LPI exams. Each part features not only a summary of the core skills you need, but sample exercises and test questions, along with helpful hints to let you focus your energies.

Major topics include:

* GNU and Unix commands
* Linux installation and package management
* Devices, filesystems, and kernel configuration
* Text editing, processing, and printing
* The X Window System
* Networking fundamentals and troubleshooting
* Security, including intrusion detection, SSH, Kerberos, and more
* DNS, DHCP, file sharing, and other networking infrastructure
* Email, FTP, and Web services

Praise for the first edition:
"Although O'Reilly's Nutshell series are intended as 'Desktop Reference' manuals, I have to recommend this one as a good all-round read; not only as a primer for LPI certification, but as an excellent introductory text on GNU/Linux. In all, this is a valuable addition to O'Reilly's already packed stable of Linux titles and I look forward to more from the author." --First Monday

# Paperback: 961 pages
# Publisher: O'Reilly Media; 2 edition (July 1, 2006)
# Language: English
# ISBN-10: 0596005288
# ISBN-13: 978-0596005283
# Product Dimensions: 9 x 6 x 2.1 inches

http://rapidshare.com/files/19002989/OReilly.LPI.Linux.Certification.in.a.Nutshell.2nd.Edition.Jul.

Mysql Backup Script

A script to take daily, weekly and monthly backups of your MySQL databases using mysqldump. Features - Backup mutiple databases - Single backup file or to a seperate file for each DB - Compress backup files - servers - remote servers - E-mail logs – More

AutoMySQLBackup Script Features

• Backup mutiple MySQL databases with one script. (Now able to backup ALL databases on a server easily. no longer need to specify each database seperately)
• Backup all databases to a single backup file or to a seperate directory and file for each database.
• Automatically compress the backup files to save disk space using either gzip or bzip2 compression.
• Can backup remote MySQL servers to a central server.
• Runs automatically using cron or can be run manually.
• Can e-mail the backup log to any specified e-mail address instead of "root". (Great for hosted websites and databases).
• Can email the compressed backup files to the specified email address.
• Can specify maximun size backup to email.
• Can be set to run PRE and POST backup commands.
• Choose which day of the week to run weekly backups.

AutoMySQLBackup Download

The AutoMySQLBackup project has now been moved to Sourceforge.net for better maintainence.

http://sourceforge.net/projects/automysqlbackup/

AutoMySQLBackup Requirements

The AutoMySQLBackup script only requires mysqldump (A standard utility provided with the mysql client) and gzip or bzip2 for compression of the backup files.

If you would like to have the log emailed to you then you will need to have permission to execute the "mail" program. If you want the compressed backup files mailed to you then Mutt must be available on the server.
Finally you will need a bash shell and the standard system tools and utilities (all these requirements should be the default on most linux system.)

What AutoMySQLBackup does

Every day AutoMySQLBackup will run (if setup on /etc/cron.daily) and using mysqldump and gzip will dump your specified databases to the /backups/daily directory, it will rotate daily backups weekly so you should never have more than 7 backups in there..

Every Saturday AutoMySQLBackup will again backup the databases you have chosen but they will be placed into /backups/weekly, these will be rotated every 5 weeks so there should never be more than 5 backups in there..

Every 1st of the month AutoMySQLBackup will create a backup of all databases and place them into /backups/monthly. These will never be rotated so it will be up to you to do your own house keeping. I would suggest taking a copy of this offline every month or two so that if you have a hard drive failure you will be able to restore your database..

AutoMySQLBackup Installation

The install is as simple as editing a few variables in the AutoMySQLBackup file. The full setup is documented in the AutoMySQLBackup script file below the variables section..

Here is a quick minimum setup step by step..

1. Download automysqlbackup.sh and place it into your /etc/cron.daily directory or your home directory.

2. Edit (at least) the following lines :-
USERNAME=dbuser (The user must have at least select privileges to the databases) PASSWORD=password
DBNAMES="DB1 DB2 DB3" (make sure to keep the quotes " " otherwise it won't work)

3. Make the file executable :- chmod u+rwx

4. Create the following directory./backups

5. That's it.. Now you can run it using the command line "./automysqlbackup.sh" or if it is in /etc/cron.daily it will run each day when cron runs.

Hack Proof Linux: A Guide to Open Source

The Linux operating system continues to gain market share based largely on its reputation as being the most secure operating system available. The challenge faced by system administrators installing Linux is that it is secure only if installed and configured properly, constantly and meticulously updated, and carefully integrated with a wide variety of Open Source security tools. The fact that Linux source code is readily available to every hacker means that system administrators must continually learn security and anti-hacker techniques.

Hack Proofing Linux will provide system administrators with all of the techniques necessary to properly configure and maintain Linux systems and counter malicious attacks.

* Linux operating systems and Open Source security tools are incredibly powerful, complex, and notoriously under-documented - this book addresses a real need
* CD-Rom contains Red Hat/Mandrake Linux RPMs as well as tarballs for other Linux distributions and the BSD community
* Uses forensics-based analysis to give the reader an insight to the mind of a hacker

Download it here

Ubuntu Guide

Ubuntu guide describing lot of How to in ubuntu.

Link: Ubuntu Guide

Set Up OpenLDAP On Fedora 7

This document describes how to set up OpenLDAP on Fedora 7. OpenLDAP is a directory server based on the LDAP protocol, that same protocol MS Active Directory is based on. OpenLDAP is an open-source implementation of the Lightweight Directory Access Protocol.

Here we go: first we install the OpenLDAP server like this: Read More ............

Basic Iptable Script

#!/bin/bash

#################################################################
#
# 1. Configuration options.
#

#################################################################
#
# Local Area Network configuration.
#
# your LAN's IP range and localhost IP. /24 means to only use the first 24
# bits of the 32 bit IP adress. the same as netmask 255.255.255.0
#

LAN_IP="192.168.0.2"
LAN_IP_RANGE="192.168.0.0/16"
LAN_BCAST_ADRESS="192.168.255.255"
LAN_IFACE="eth1"

################################################################
#
# Localhost Configuration.
#

LO_IFACE="lo"
LO_IP="127.0.0.1"

################################################################
#
# Internet Configuration.
#

INET_IP="194.236.50.155"
INET_IFACE="eth0"

################################################################
#
# IPTables Configuration.
#

IPTABLES="/usr/sbin/iptables"

################################################################
#
# 2. Module loading.
#

#
# Needed to initially load modules
#
/sbin/depmod -a

#
# Adds some iptables targets like LOG, REJECT and MASQUARADE.
#
/sbin/modprobe ip_conntrack
/sbin/modprobe ip_tables
/sbin/modprobe iptable_filter
/sbin/modprobe iptable_mangle
/sbin/modprobe iptable_nat
/sbin/modprobe ipt_LOG
#/sbin/modprobe ipt_REJECT
#/sbin/modprobe ipt_MASQUERADE

#
# Support for owner matching
#
#/sbin/modprobe ipt_owner

#
# Support for connection tracking of FTP and IRC.
#
#/sbin/modprobe ip_conntrack_ftp
#/sbin/modprobe ip_conntrack_irc


###############################################################
#
# 3. /proc set up.
#
# Enable ip_forward if you have two or more networks, including the
# Internet, that needs forwarding of packets through this box. This is
# critical since it is turned off as default in Linux.
#

echo "1" > /proc/sys/net/ipv4/ip_forward

#
# Dynamic IP users:
#
#echo "1" > /proc/sys/net/ipv4/ip_dynaddr

################################################################
#
# 4. IPTables rules set up.
#
# Set default policies for the INPUT, FORWARD and OUTPUT chains.
#

$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT DROP
$IPTABLES -P FORWARD DROP

#
# bad_tcp_packets chain
#
# Take care of bad TCP packets that we don't want.
#

$IPTABLES -N bad_tcp_packets
$IPTABLES -A bad_tcp_packets -p tcp ! --syn -m state --state NEW -j LOG \
--log-prefix "New not syn:"
$IPTABLES -A bad_tcp_packets -p tcp ! --syn -m state --state NEW -j DROP

#
# Do some checks for obviously spoofed IP's
#

$IPTABLES -A bad_tcp_packets -i $INET_IFACE -s 192.168.0.0/16 -j DROP
$IPTABLES -A bad_tcp_packets -i $INET_IFACE -s 10.0.0.0/8 -j DROP
$IPTABLES -A bad_tcp_packets -i $INET_IFACE -s 172.16.0.0/12 -j DROP

#
# Enable simple IP Forwarding and Network Address Translation
#

$IPTABLES -t nat -A POSTROUTING -o $INET_IFACE -j SNAT --to-source $INET_IP

#
# Bad TCP packets we don't want
#

$IPTABLES -A FORWARD -p tcp -j bad_tcp_packets

#
# Accept the packets we actually want to forward
#

$IPTABLES -A FORWARD -i $LAN_IFACE -j ACCEPT
$IPTABLES -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -m limit --limit 3/minute --limit-burst 3 -j LOG \
--log-level DEBUG --log-prefix "IPT FORWARD packet died: "

#
# Create separate chains for ICMP, TCP and UDP to traverse
#

$IPTABLES -N icmp_packets
$IPTABLES -N tcp_packets
$IPTABLES -N udpincoming_packets

#
# The allowed chain for TCP connections
#

$IPTABLES -N allowed
$IPTABLES -A allowed -p TCP --syn -j ACCEPT
$IPTABLES -A allowed -p TCP -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A allowed -p TCP -j DROP

#
# ICMP rules
#

# Changed rules totally
$IPTABLES -A icmp_packets -p ICMP -s 0/0 --icmp-type 8 -j ACCEPT
$IPTABLES -A icmp_packets -p ICMP -s 0/0 --icmp-type 11 -j ACCEPT

#
# TCP rules
#

$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport 21 -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport 22 -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport 80 -j allowed
$IPTABLES -A tcp_packets -p TCP -s 0/0 --dport 113 -j allowed

#
# UDP ports
#

# nondocumented commenting out of these rules
#$IPTABLES -A udpincoming_packets -p UDP -s 0/0 --source-port 53 -j ACCEPT
#$IPTABLES -A udpincoming_packets -p UDP -s 0/0 --source-port 123 -j ACCEPT
$IPTABLES -A udpincoming_packets -p UDP -s 0/0 --source-port 2074 -j ACCEPT
$IPTABLES -A udpincoming_packets -p UDP -s 0/0 --source-port 4000 -j ACCEPT

##########################
# INPUT chain
#
# Bad TCP packets we don't want.
#

$IPTABLES -A INPUT -p tcp -j bad_tcp_packets

#
# Rules for incoming packets from the internet.
#

$IPTABLES -A INPUT -p ICMP -i $INET_IFACE -j icmp_packets
$IPTABLES -A INPUT -p TCP -i $INET_IFACE -j tcp_packets
$IPTABLES -A INPUT -p UDP -i $INET_IFACE -j udpincoming_packets

#
# Rules for special networks not part of the Internet
#

$IPTABLES -A INPUT -p ALL -i $LAN_IFACE -d $LAN_BCAST_ADRESS -j ACCEPT
$IPTABLES -A INPUT -p ALL -i $LO_IFACE -s $LO_IP -j ACCEPT
$IPTABLES -A INPUT -p ALL -i $LO_IFACE -s $LAN_IP -j ACCEPT
$IPTABLES -A INPUT -p ALL -i $LO_IFACE -s $INET_IP -j ACCEPT
$IPTABLES -A INPUT -p ALL -i $LAN_IFACE -s $LAN_IP_RANGE -j ACCEPT
$IPTABLES -A INPUT -p ALL -d $INET_IP -m state --state ESTABLISHED,RELATED \
-j ACCEPT
$IPTABLES -A INPUT -m limit --limit 3/minute --limit-burst 3 -j LOG \
--log-level DEBUG --log-prefix "IPT INPUT packet died: "

###############################
# OUTPUT chain
#
#
# Bad TCP packets we don't want.
#

$IPTABLES -A OUTPUT -p tcp -j bad_tcp_packets

#
# Special OUTPUT rules to decide which IP's to allow.
#

$IPTABLES -A OUTPUT -p ALL -s $LO_IP -j ACCEPT
$IPTABLES -A OUTPUT -p ALL -s $LAN_IP -j ACCEPT
$IPTABLES -A OUTPUT -p ALL -s $INET_IP -j ACCEPT

#
# Log weird packets that don't match the above.
#

$IPTABLES -A OUTPUT -m limit --limit 3/minute --limit-burst 3 -j LOG \
--log-level DEBUG --log-prefix "IPT OUTPUT packet died: "

Basic backup script using tar

If you are looking for a simple backup script using tar then the below script is enough.

*************************************************************************************
#!/bin/sh
# full and incremental backup script
# created 07 February 2000
# Based on a script by Daniel O'Callaghan 
# and modified by Gerhard Mourani 

#Change the 5 variables below to fit your computer/backup

COMPUTER=deep                            # name of this computer
DIRECTORIES="/home"                      # directoris to backup
BACKUPDIR=/backups                       # where to store the backups
TIMEDIR=/backups/last-full               # where to store time of full backup
TAR=/bin/tar                              # name and locaction of tar

#You should not have to change anything below here

PATH=/usr/local/bin:/usr/bin:/bin
DOW=`date +%a`                # Day of the week e.g. Mon
DOM=`date +%d`                # Date of the Month e.g. 27
DM=`date +%d%b`              # Date and Month e.g. 27Sep

# On the 1st of the month a permanet full backup is made
# Every Sunday a full backup is made - overwriting last Sundays backup
# The rest of the time an incremental backup is made. Each incremental
# backup overwrites last weeks incremental backup of the same name.
#
# if NEWER = "", then tar backs up all files in the directories
# otherwise it backs up files newer than the NEWER date. NEWER
# gets it date from the file written every Sunday.


# Monthly full backup
if [ $DOM = "01" ]; then
    NEWER=""
    $TAR $NEWER -cf $BACKUPDIR/$COMPUTER-$DM.tar $DIRECTORIES
fi

# Weekly full backup
if [ $DOW = "Sun" ]; then
    NEWER=""
    NOW=`date +%d-%b`

    # Update full backup date
    echo $NOW > $TIMEDIR/$COMPUTER-full-date
    $TAR $NEWER -cf $BACKUPDIR/$COMPUTER-$DOW.tar $DIRECTORIES

# Make incremental backup - overwrite last weeks
else

    # Get date of last full backup
    NEWER="--newer `cat $TIMEDIR/$COMPUTER-full-date`"
    $TAR $NEWER -cf $BACKUPDIR/$COMPUTER-$DOW.tar $DIRECTORIES
fi
*************************************************************************************