Firstly, we can use the following query to generate a list of mysqldump commands. The backups generated here pretty much just contain tables, triggers and functions so customise the command to your needs. Just change the output directory from ./home/rhys.

SELECT CONCAT('mysqldump -d --routines -h mysql_host -u username -pSecret ', SCHEMA_NAME, ' > "/home/rhys/Desktop/', SCHEMA_NAME, '.sql"')
FROM INFORMATION_SCHEMA.SCHEMATA
WHERE SCHEMA_NAME NOT IN ('mysql', 'information_schema', 'performance_schema');

These commands can be copied and then pasted into a terminal to be executed.

Here’s a pure bash way of achieving the same thing. The script is called on the command line with the MySQL host, user name and password supplied as parameters. Backups are generated in the current users home directory in the format db_name.sql.

#!/bin/bash
# MySQL details passed on command line
HOST=$1
USER=$2;
PWD=$3;
 
# Query to get database excluding a few system ones
QUERY="SELECT SCHEMA_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME NOT IN ('mysql','information_schema','performance_schema')";
# Run the query and get the results
results=`mysql -h $HOST -u $USER -p$PWD -N -e "$QUERY"`;
 
# Loop through each row
for db in $results
do
	bkp="mysqldump -d --routines -h $HOST -u $USER -p$PWD $db > '$HOME/$db.sql'";
	echo "Backing up $db...";
	eval $bkp;
done

Save this to a file called backup_mysql.sh and don;t forget to do a chmod + x to make the script executable. The backup can be executed from the command line with;

./backup_mysql.sh localhost username password

Post from: youdidwhatwithtsql.com

Backing up the structure of MySQL databases

mysqlsla parses, filters, analyzes and sorts MySQL slow, general, binary and microslow patched logs in order to create a customizable report of the queries and their meta-property values. Since these reports are customizable, they can be used for human consumption or be fed into other scripts to further analyze the queries. For example, to profile with mk-query-profiler (a script from Baron Schwartz’s Maatkit) every unique SELECT statement using database foo from a slow log: source

Place all your slow logs into a directory. Change the sl_dir variable to point at this directory. When you execute the script it will create a directory, within your slow logs directory, called reports. This will contain the reports produced by mysqlsla.

#!/bin/bash
 
# Script to process multiple mysql slow logs
# using mysqlsla http://hackmysql.com/mysqlsla
 
# Directory containing slow logs
sl_dir="/home/rhys/Desktop/slow_logs";
 
cd "$sl_dir";
#slow_logs=$(ls "$sl_dir");
 
# Folder for reports
if [ ! -d "$sl_dir"/reports ]; then
                mkdir "$sl_dir"/reports;
fi
 
# process each slow log file
for file in "$sl_dir"/*
do
                echo "Processing file: $file";
                filename=$(basename "$file")
                mysqlsla -lt slow "$file" > "reports/$filename.rpt";
                echo "Finished processing file: $file";
done

The reports produced are much easier to work with than the raw mysql logs so this should be a good time saver when optimising those queries!

Post from: youdidwhatwithtsql.com

Parse MySQL Slow Logs with mysqlsla

Save the script below to your home directory called chk_dsk_space.sh and mark this as executable. Open a terminal and enter ./chk_dsk_space.sh. You will be prompted for the password for each server before it displays disk usage information.

#!/bin/bash
 
# Set computer names here to check
computers="user@server1 user@server2 user@server3"
# Work through each computer in the array
for c in $computers
do
                echo "================================================="
                echo "= $c"
                echo "================================================="
                command="ssh $c 'df -h'"
                eval $command
done

Here’s some sample output.

rhys@linux-n0sm:~> ./chk_dsk_space.sh
=================================================
= rhys@server1
=================================================
Password:
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda6             6.7G  4.2G  2.2G  67% /
udev                  1.7G  444K  1.7G   1% /dev
/dev/sda7             8.9G  417M  8.0G   5% /home
/dev/sda2              75G   71G  4.1G  95% /windows/C
/dev/sda3              55G   47G  7.6G  86% /windows/D
=================================================
= user@server2
=================================================
Password:
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda6             6.7G  4.2G  2.2G  67% /
udev                  1.7G  444K  1.7G   1% /dev
/dev/sda7             8.9G  417M  8.0G   5% /home
/dev/sda2              75G   71G  4.1G  95% /windows/C
/dev/sda3              55G   47G  7.6G  86% /windows/D
=================================================
= user@server3
=================================================
Password:
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda6             6.7G  4.2G  2.2G  67% /
udev                  1.7G  444K  1.7G   1% /dev
/dev/sda7             8.9G  417M  8.0G   5% /home
/dev/sda2              75G   71G  4.1G  95% /windows/C
/dev/sda3              55G   47G  7.6G  86% /windows/D

Not as elegant as my Powershell script but it’s functional and saves me a little time each day. I’m not finding Bash as easy to work with as Powershell but that’s probably more due to my lack of experience than anything else. There’s an Open Source implementation of Powershell I’ve been thinking of checking out, it doesn’t implement the Windows specific cmdlets, but it’s been good to look at another way of doing things with Bash.

Post from: youdidwhatwithtsql.com

Check disk space with Bash

First thing that might trip you up is the Execution Policy built into Powershell. I’ve turned this off on my development machine but it’s obviously advised to have it enabled on production machines. That aside, it’s easy to turn off…

If you’re using Vista you may need to run Powershell in Administrator mode to allow the required registry change. To make my environment a little more user friendly I’ve setup a Powershell Profile with the following code

cd C:\Users\Rhys\Documents\powershell
function prompt {"PS: $(get-date)>"}

This changes the current working directory to my scripts folder and keeps the prompt reasonably short.

Write text files with Powershell

Just look how easy it is to write text files in Powershell.

$text = "abcdefghijklmnopqrstuvwxyz1234567890"
 
# Method 1: Write file using set-content. 
# Just pipe the contents of the variable
$text | set-content -encoding ascii alphabet.txt
 
# Method 2: Use out-file to append the for loop index to a file
for($i = 0; $i -lt 10; $i++)
{
	$i | out-file -filePath "forLoop.txt" -encoding ascii -append 
}

After executing this you should have 2 new .txt files in your working directory.

List & Count Files by Extension with Powershell

This script will list filenames and their sizes before displaying a total count of those files. Just change the extension in the script if you want to work with different file types.

#####################################################
# Filename: DirectoryTextFiles.ps1		    #
# Author: Rhys Campbell				    #
# Description: Lists all .txt files in the current  #
# directory, with their length in bytes, and        #
# displays a total count of .txt files.		    #
# Date: 2009-04-22				    #
#####################################################
 
$count = 0;
 
$files = dir *.txt
 
foreach($file in $files)
{
	$count++;
	$filename = $file.name
	$fileLength = $file.Length 
	echo "The text file $filename is $fileLength bytes.";
}
 
echo "===================================================="
echo "Total text files in this directory = $count";

Get the Twitter Public Timeline with Powershell

One line of code is all is takes with Powershell!

([xml](new-object net.webclient).DownloadString("http://twitter.com/statuses/public_timeline.rss")).rss.channel.item | format-table title,link

Ping multiple computers with Powershell

###############################################
# Filename: ComputerPinger.ps1		      #
# Author: Rhys Campbell			      #
# Description: Pings multiple computers       #
# supplied in the $args array.		      #
# Date: 2009-04-16			      #
###############################################
 
# Simple ping for each computer
# supplied in the $args array
foreach($computer in $args)
{
	ping $computer;
}

Count Databases On Multiple Servers

Those of you that are still with me get to see the exciting stuff. This script will count databases, and see if SQLAGENT is running, on multiple servers. Just specify each server on the command line. This script uses Windows authentication so bear this in mind for each of your servers. It’s with examples like this that I hope you start to see the power and possibilities of Powershell.

################################################################################
# Filename: MultiHostDatabaseCount.ps1					       #
# Description: 								       #
# Author: Rhys Campbell							       #
# Date: 2009-04-15							       #
# Usage: FullPathToFile\MultiHostDatabaseCount.ps1 server1 server2 server3 etc #
################################################################################
 
trap
{
	"Eek! An exception occured."
	write-error $("TRAPPED: " + $_.Exception.GetType().FullName);
	write-error $("TRAPPED: " + $_.Exception.Message); 
	# Terminate the script on error
	exit
}
 
# Loop through each server name supplied in the $args array
foreach($arg in $args)
{
	# Connection string
	$connectionString = "Server=$arg;Database=master;Trusted_Connection=True;"
	# setup the connection object
	$SqlConn = New-Object -Typename System.Data.SqlClient.SqlConnection
	$SqlConn.ConnectionString = $connectionString
	# Open the connection
	$SqlConn.open();
	$cmd = "SELECT COUNT(*) AS cnt FROM sys.databases";
	# Create the sql command object
	$SqlCmd = New-Object System.Data.SqlClient.SqlCommand $cmd, $SqlConn
	# Run the query
	$result = $SqlCmd.ExecuteReader()
	# Move pointer to the first row
	$result.Read() > $null
	# Get db count
	$dbCount = $result[0]
	# Close the connection
	$SqlConn.close()
	# Reopen the connection
	$SqlConn.open()
	# New command to get SQLAGENT state
	$cmd = "xp_servicecontrol 'querystate', 'SQLSERVERAGENT'"
	# Create the sql command object
	$SqlCmd = New-Object System.Data.SqlClient.SqlCommand $cmd, $SqlConn
	# Run the query
	$result2 = $SqlCmd.ExecuteReader()
	# Move pointer to the first row
	$result2.Read() > $null
	# Get db count
	$SQLAGENT = $result2[0]
	# Print out the database count for the host
	echo "$arg hosts $dbCount databases. The SQLAGENT Service is $SQLAGENT"
	# Clean up
	$SqlConn.close()
}

I hope to check out the huge range of Powershell Extensions and Tools available to do some really cool things. I’ve had a quick play with Quest Software’s PowerGUI. This looks pretty good but I try to stay away from fancy tools until I’ve learned the basics of a language well. I’m looking forward to working with Powershell scripting. I see some great possibilities for…

• Multiple SQL Server & MySQL management. Check multiple servers in one script! Check backups, disk space, virtually anything!
• Adhoc access to many data sources; including text files, rss feeds, databases, Active Directory. 
• Producing troubleshooting scripts for less experienced support staff.

Post from: youdidwhatwithtsql.com

Powershell Nuggets