~ overflow ~

Problem:

So i have this shitty server that keeps overloading, although i have all sort of monitors around, many screens, etc I really can’t keep looking at the graphs all day long, but yet i’m almost of the time on the computer.

So why not have a sound trigger when server load go over 10?

Solution:

This php script solves it:

// (c) z3n - R1V1@111227 - www.overflow.biz - rodrigo.orph@gmail.com

while (true) {
	$output = explode(" ", trim(substr(strstr(exec("uptime"), "load average:"), 13)));
        if (substr($output[0], 0, -1) > 10)
		@ passthru("echo -e -n \\\\a");

	echo "\r[" . date("Y-m-d H:i:s") . "] " . implode(" ", $output);
	sleep(5);
}

I wrote this macro with XTE intended to run on linux on a VNC screen, nothing a regular geek wouldn’t understand.

Never loose a poke war ever again:

while [ 1 ] ; do xte 'mousemove 20 350' 'mouseclick 1';sleep 5;xte 'mousemove 300 235' 'mouseclick 1';sleep 30;done;

Problem:

FTPd is working however when you try to login you get access denied on user’s home folder, no matter what accesses and group/user combination.

Solution:
Check /var/log/audit/autit.log and /var/log/messages
This usually is a SELinux block, fix it using:

 setsebool -P ftp_home_dir=1

Problem:

How to change sendmail’s outgoing ip?

Solution:

This is the ip that will be used to deliver messages, this ip might get blacklisted if one of your happy clients on a shared server decides to send a mass email.
Sendmail is always a bitch to config, but this time it wasn’t, just change your .m4 file adding:

CLIENT_OPTIONS(`Addr=123.456.789')dnl

rebuild the .cf file, restart sendmail and you’re done. In my case, I had a specific ip on the server that got “temporary” (permanently) blocked by gmail. Although new mail wasn’t being sent after the flood, google never unblocked my server, even after a week i still getting “limit rate exceeded” messages. Since I have many IPs on the server it was easier just to bind sendmail to another IP.

Source:

Server Fault

Problem:

MySQL InnoDB ibdata file size never shrinks no matter if you deleted a big database / table or whatever.

Solution:

There is no easy solution for this, according to mysql this is supposed to be a bug, however they never fixed it since, apparently, there’s not many people around deleting big tables / databases.

Looking around, someone suggested that recreating the whole dataset would help fixing the issue, since we won’t be having trash on the innodb ibdata file.

Recreating the whole dataset is a extremely time consuming task, you would need to do a dump of every database, completly delete mysql data folder, create a new one with defaults, and inject the dump file. Just for example, on my dev machine i run about 100 different databases the total size is around 8GB, innodb was 20GB, wasting a lot of space on my precious SSD, so dumping everything, doing backups and injecting the whole data took over 8 hours, which is something nobody can take on a production server.

After some more research I’ve found about a specific configuration switch:

innodb_file_per_table

This is what you should have on your my.cnf or my.ini into the innodb branch. Instead of creating one big innodb ibdata for every single innodb table across all databases, this switch will create a `.ibd` file for each of the the innodb tables. Those files will be stored inside the database folder, so if you eventually have a greatly sized table you just need to drop it and recreate the single table to avoid gigantic innodb .ibd files instead of doing a full database recreation.

If you don’t have time to recreate everything you can just add `innodb_file_per_table` into the config, so new tables will be created using the new scheme.

The only con i can think about this switch is because you will have one extra file that will have to be seeked and will incrase the overhead when doing table joins and so on, although the disk overhead should be minimal, if you have thousands of tables into the same db it might make some difference, if you relay on speed you should do some benchmarks to make sure.

Bug Tracking:

bugs.mysql.com

Problem:

How to find big files on linux?

Solution:

 find / -type f -size +50000k

This will find files bigger than 50MB (50000k)

Problem:
How to convert a generic object into an array?

Solution:
I’ve wrote this to convert simple xml objects into array, works pretty good.

function toArray($obj) {
	if (is_object($obj))
		$obj = (array)$obj;

	if (is_array($obj)) {
		foreach ($obj as &$val)
			$val = toArray($val);
	} else { // if this is not an array so it's a string
		$obj = (string)$obj;
	}

	return $obj;
}

Problem:

How to validate an email using php?

Solution:

I’m posting this function because i made it to have older php compatibility.

function email_validator($email) {
	return function_exists("filter_var") ?
			filter_var($email, FILTER_VALIDATE_EMAIL)
		: // if filter_var is not defined (php < 5.2), we will emulate php's core here
			!empty($email) && strlen($email) <= 320 &&
			/**
			 * preg info:
			 *
			 * Copyright © Michael Rushton 2009-10
			 * http://squiloople.com/
			 *
			 * more at: http://svn.php.net/viewvc/php/php-src/trunk/ext/filter/logical_filters.c?view=markup
			 */
			preg_match("/^(?!(?:(?:\\x22?\\x5C[\\x00-\\x7E]\\x22?)|(?:\\x22?[^\\x5C\\x22]\\x22?)){255,})(?!(?:(?:\\x22?\\x5C[\\x00-\\x7E]\\x22?)|(?:\\x22?[^\\x5C\\x22]\\x22?)){65,}@)(?:(?:[\\x21\\x23-\\x27\\x2A\\x2B\\x2D\\x2F-\\x39\\x3D\\x3F\\x5E-\\x7E]+)|(?:\\x22(?:[\\x01-\\x08\\x0B\\x0C\\x0E-\\x1F\\x21\\x23-\\x5B\\x5D-\\x7F]|(?:\\x5C[\\x00-\\x7F]))*\\x22))(?:\\.(?:(?:[\\x21\\x23-\\x27\\x2A\\x2B\\x2D\\x2F-\\x39\\x3D\\x3F\\x5E-\\x7E]+)|(?:\\x22(?:[\\x01-\\x08\\x0B\\x0C\\x0E-\\x1F\\x21\\x23-\\x5B\\x5D-\\x7F]|(?:\\x5C[\\x00-\\x7F]))*\\x22)))*@(?:(?:(?!.*[^.]{64,})(?:(?:(?:xn--)?[a-z0-9]+(?:-[a-z0-9]+)*\\.){1,126}){1,}(?:(?:[a-z][a-z0-9]*)|(?:(?:xn--)[a-z0-9]+))(?:-[a-z0-9]+)*)|(?:\\[(?:(?:IPv6:(?:(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){7})|(?:(?!(?:.*[a-f0-9][:\\]]){7,})(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,5})?::(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,5})?)))|(?:(?:IPv6:(?:(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){5}:)|(?:(?!(?:.*[a-f0-9]:){5,})(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,3})?::(?:[a-f0-9]{1,4}(?::[a-f0-9]{1,4}){0,3}:)?)))?(?:(?:25[0-5])|(?:2[0-4][0-9])|(?:1[0-9]{2})|(?:[1-9]?[0-9]))(?:\\.(?:(?:25[0-5])|(?:2[0-4][0-9])|(?:1[0-9]{2})|(?:[1-9]?[0-9]))){3}))\\]))$/iD", $email)
	;
}

Problem:

You need to find a entry or a specific value that’s into a table from a database, but you don’t know what is the table neither the database for sure.

Solution:

Use my happy mysql full database search script.
It’s so fun when all you gotta do is type a single command.

download mysql full database search script

Problem:

You did a svn checkout on this nice project but now you don’t want that .svn folders all over the place, after all you won’t be commiting it. Suddenly you find out that there’s thousands of folders with thousands of files.

Solution:

I wrote this little php cli script to fix this:

<?php

// (c) z3n - R1V1@101203 - www.overflow.biz - rodrigo.orph@gmail.com

function _o($msg) {
	echo "\r" . substr($msg,0, 79) . str_repeat(" ", 79 - strlen($msg));
}

function rddir($dir) { /* v2.17 */
	global $tfile,$tdir;

	if (is_dir($dir)) {
		if ($dh = opendir($dir)) {
			while (($file = readdir($dh)) !== false) {
				if ($file != "." && $file != "..") {
					if (is_dir($dir . $file))
						$tdir[] = $dir . $file;
					else
						$tfile[] = $file;
				}
			}
			closedir($dh);
		}
	}
}

if (!isset($argv[1]))
	die("Usage: " . $_SERVER['PHP_SELF'] . " <path/to/svn_root/>");

// browse dirs searching for .svn

for (
	$tdir = array($argv[1]),
	$tfile = array(),
	$svn = array(),
	$i = 0;$i < count($tdir);$i++)
{
	rddir($tdir[$i]);
	_o("Reading: ".$tdir[$i]." ...");

	foreach ($tfile as $entry) {
		if ($entry != "." && $entry != "..") {
			$fn = $tdir[$i] . (PHP_OS == "WINNT" ? "\\" : "/") . $entry;
			if ($entry == ".svn" && !in_array($fn, $svn))
				$svn[] = $fn;

			if (is_dir($fn) && $entry != ".svn")
				$tdir[] = $fn;
		}
	}

	$tfile = array();
}

_o("Found: " . count($svn) . " .svn folders, collecting files...");

$files = array();
$dirs = array();
$t = 0;

// collect files from .svn folders

foreach ($svn as $dir) {
	for ($tdir = array($dir), $tfile = array(),$i = 0;$i < count($tdir);$i++) {
		rddir($tdir[$i]);
		_o("[" . $t . "] - Collecting: ". $tdir[$i] ."...");

		foreach ($tfile as $entry) {
			if ($entry != "." && $entry != "..") {
				$fn = $tdir[$i] . (PHP_OS == "WINNT" ? "\\" : "/") . $entry;
				$t++;

				if (is_dir($fn)) {
					$tdir[] = $fn;
					$dirs[] = $fn;
				} else {
					$files[] = $fn;
				}
			}
		}
	}
}

$i = count($files) + count($dirs) + count($svn);

// delete files

foreach ($files as $file) {
	_o("[" . $i . "] Deleting Files: " . $file);
	unlink($file);
	$i--;
}

// delete subfolders

rsort($dirs);

foreach ($dirs as $dir) {
	_o("[" . $i . "] Deleting Dir: " . $dir);
	rmdir($dir);
	$i--;
}

// delete root .svn folders

foreach ($svn as $dir) {
	_o("[" . $i . "] Deleting Roots: " . $dir);
	rmdir($dir);
	$i--;
}

echo "\nDone!";

?>