Happy sysadmin day to my fellow colegues! I hope we will have a ‘quiet’ weekend to celebrate this properly, and not have to deal with any unexpected emergency…

Here’s a little post on how to change the master database host that the replication slave in a MySQL replication setup uses. This can happen if there is a change in network addresses or when you want to switch over to using different internal network. The main thing to keep in mind is that when you change the replication user host on the slave MySQL will think that you are now replicating from a different database. Therefore we need to force MySQL to continue replicating the changes where it left off before the change.

First we need to create a new user on the master database server with the new IP address like this:

1

GRANT REPLICATION SLAVE ON *.* TO 'repl'@'10.0.0.1' IDENTIFIED BY 'password';

The rest of the operations are on the replication slave. We need to stop the replication and figure out where to restart the replication after changing the user:

1
2

STOP SLAVE;
SHOW SLAVE STATUS

From the slave status pick out the 2 settings for Master_Log_File and Read_Master_Log_Pos. Those items will look something like this:

1
2

Master_Log_File: mysql-bin.000101
Read_Master_Log_Pos: 591523680

Those settings need to be used in the following statement:

1
2

CHANGE MASTER TO MASTER_HOST='10.0.0.1', MASTER_LOG_FILE='mysql-bin.000101', MASTER_LOG_POS=591523680;
START SLAVE;

The start slave command starts the replication again. Make sure you do a show slave status afterwards to make sure that replication is running again without errors.

Issue: Apache does not start, gives the following error

1

965 File size limit exceeded$HTTPD -DSSL

Cause: Apache Log or Domain Log file exceeded 2GB

Solution: Rotate offending log file by tar/gzip’ing

Issue: Hosting account do not show up in WHM

Cause: /etc/trueuserdomains file which holds a list of account on the server is empty

Solution: restore contents of /etc/trueuserdomains and update cpanel with /scripts/upcp

Very funny: http://www.thewebsiteisdown.com/

:-)

The error message in the title occurs in a combination of circumstances. If you have 2 MySQL databases, one master and one slave in a replicating setup and you use triggers you may encounter this error. Although it is not necessary for the slave database to have the same users as the master for replication to work it is required for triggers to work. The triggers get fired at both the master and the slave and use the username of the user that caused the trigger to execute. So if user user@localhost is present at the master then the same user needs to be added to the slave. The privileges don’t have to be the same as long as the user has enough access on the slave to complete the trigger.

The biggest problem with this is that it causes replication to stop dead in its tracks. After adding the user to the slave database the replication needs to be restarted with the following MySQL command:

1

START SLAVE;

After that check for errors with the following MySQL command:

1

SHOW SLAVE STATUS

Depending on how long replication has been suspended it may take some time for the slave to catch up. This will not be shown in the replication status.

The CentOS team just released their latest version of CentOS-5.2, a minor upgrade for the Centos-5 release. This is based on the upstream release RHEL 5.2.0 and includes packages from all variants including server and client repositories.

Any system running CentOS-5 (either 5.0 or 5.1) can be upgraded by just running: yum upgrade that will bring in the new packages from 5.2 and update the system. Depending on the size of your install and your connection speed this can take a while to complete ;-) .

This post describes a way to implement data encryption in the front-end of an application and how to make it compatible between a PHP front-end and a Java front-end.

First off we need to decide what type of encryption to use. AES is an encryption standard that has been endorsed by the NSA and is generally available for most programming languages. It is actually the same as the Rijndael encryption scheme but with a couple of limitations. Rijndael describes a two-way encryption scheme which allows us to encrypt and decrypt data using an encryption key and a random generator. The random generator is used to make sure that when we’re encrypting two identical values the resulting encrypted values will be different. Since we’re encrypting credit card data like the expiration date this is actually a very desirable feature. The random value is called the Initialization Vector (I.V.) and will be stored with the encrypted data. To successfully encrypt and decrypt data we will need information stored in 3 different locations:

1. The encryption key which is stored on the web server in a plain text file
2. The encryption key hash function which is stored in the front-end code
3. The encrypted data and the I.V. which are both stored in the database

Both the encryption key and the hash function are needed to encrypt data. To decrypt data the I.V. and the encrypted data are also needed of course. If any of these items is not available decryption can not take place. The hash function for the encryption key is used to make sure that the key cannot be compromised by just retrieving the key file. The key in the key file needs to be encrypted with an MD5 based function to be useful.

For the second part I will describe how to encrypt and decrypt data using this method in PHP. The main concern here is to use only functions and parameters that are compatible with the Java front-end. As it turns out some PHP functions are not quite as generic as they appear at first glance. The crypt() function is a good example of this. Although the crypt() function has MD5 functionality it is by no means generic. The value is encrypted several times and a random value (a salt) is inserted in several locations. With these variations being buried in the PHP source code it is hard to match the same encryption in Java. On the other hand the md5() function does provide a straightforward implementation of MD5 encryption. It is advisable to apply MD5 several times and insert a hard coded value into the string. This is similar to what the crypt() function does but by doing it ourselves we can make sure of compatibility between the platforms as well as add some security by making it custom.

After encryption the key is ready for use and it is time to encrypt or decrypt the data. PHP comes with the mcrypt library that supports Rijndael encryption with I.V’s (installation of the mcrypt library is described in a previous post). The limitation of using the mcrypt library is that the length of the encryption key and the length of the I.V. must be identical. Furthermore there is no way to specify what method of padding to use. When data is encrypted it will always result in blocks of data of the same size. If the data does not fill up the block completely it is padded with empty data. The default padding method is TBC. The final code for encrypting a string will look something like this:

1
2

$iv = mcrypt_create_iv(mcrypt_get_iv_size(MCRYPT_RIJNDAEL_256, MCRYPT_MODE_CBC), MCRYPT_RAND);
$encrypted_string = mcrypt_encrypt(MCRYPT_RIJNDAEL_256, $key, $input_string, MCRYPT_MODE_CBC, $iv);

The final part of this post will deal with how to implement the same encryption in Java. The first problem we run into is trying to implement this with the standard Java encryption extensions.

First of all the 256 bits version of Rijndael is not supported without the Unlimited Strength Jurisdiction Policy files. These can be downloaded from the main JDK download page at the Sun web site. But even with these files installed we still have some problems like no support for initialization vectors. This means we have to turn to alternative libraries like the gnu-crypto library. This library contains everything we need for compatibility. It allows for the use of initialization vectors, TBC padding and different cypher block modes. We have not touched upon the use of cypher block modes but they define how blocks of data are encrypted. Which mode to use is largely dependent on the type of data will be encrypted and it’s definitely outside the scope of this post to delve into that. Originally we started using ECB mode with the PHP front-end but found that that mode is incompatible with the gnu-crypto implementation in Java. Instead we now use the CBC mode.

The Java code is limited to the use of the encryption function but will need to use the IPad class afterwards to add the correct padding.

1
2
3
4
5
6
7

IMode mode = ModeFactory.getInstance("CBC", "Rijndael", 32);
Map attributes = new HashMap();
attributes.put(IMode.KEY_MATERIAL, key);
attributes.put(IMode.CIPHER_BLOCK_SIZE, new Integer(32));
attributes.put(IMode.STATE, new Integer(IMode.ENCRYPTION));
attributes.put(IMode.IV, iv);
mode.init(attributes);

Some final tips:

• Insert a special string with each encrypted string to make it possible to determine whether a string has been encrypted or not.
• Calculate how big the data will be based on the block sizes and adjust the database columns accordingly. Using 256 bits encryption is very secure but it results in a large block size which is very inefficient for short data.
• If you’re upgrading your application to use encryption provide the developers with some easy functions to insert and upgrade the encrypted data.

FFMPEG is an open source application that allows you to convert video and audio files easily between a variety of different formats. It supports most industry-standard codec and can convert from one file format to another quickly and easily. This guide is intented for the installatiion of ffmpeg, ffmpeg-php, mplayer, mencoder, lame mp3 encoder, flvtool2, libVorbis, and libogg and tested on CentOS5 and RHEL3 systems with Cpanel.

To start:

1

cd /usr/src

Download MPlayer and Codes, FlvTool2, Lame, and ffmpeg-php:

MPlayer codecs (check for latest release):

1

wget http://www3.mplayerhq.hu/MPlayer/releases/codecs/essential-20071007.tar.bz2

MPlayer (check for latest release):

1

wget http://www4.mplayerhq.hu/MPlayer/releases/MPlayer-1.0rc2.tar.bz2

FlvTool2 (check for latest release):

1

wget http://rubyforge.org/frs/download.php/17497/flvtool2-1.0.6.tgz

Lame MP3 Encoder (check for latest release):

1

wget http://easynews.dl.sourceforge.net/sourceforge/lame/lame-3.97.tar.gz

FFMPEG-PHP (check for latest release):

1

wget http://nchc.dl.sourceforge.net/sourceforge/ffmpeg-php/ffmpeg-php-0.5.3.1.tbz2<

Extract downloaded files:

1
2
3
4
5

tar -zxvf flvtool2-1.0.6.tgz
tar -zxvf lame-3.97.tar.gz
bunzip2 essential-20071007.tar.bz2;  tar xvf essential-20071007.tar
bunzip2 MPlayer-1.0rc2.tar.bz2; tar xvf MPlayer-1.0rc2.tar
bunzip2 ffmpeg-php-0.5.3.1.tbz2; tar xvf ffmpeg-php-0.5.3.1.tar

Create and import the Codecs directory:

1
2
3

mkdir /usr/local/lib/codecs/
mv essential-20071007/* /usr/local/lib/codecs/
chmod -Rf 755 /usr/local/lib/codecs/

Install Subversion and Ruby using Yum:

1

yum install subversion

You may need to open SVN port 3690

1

yum install ruby

If you’re on cPanel you can alternatively use /scripts/installruby

1

yum install ncurses-devel

Get FFMPEG and MPlayer from SVN:

1
2

svn checkout svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer

Install LAME:

1
2

cd /usr/src/lame-3.97
./configure && make && make install

Install libOgg and libVorbis:

1

yum install libogg.i386 libvorbis.i386 libvorbis-devel.i386

Install flvtool2:

1
2
3
4

cd /usr/src/flvtool2-1.0.6/
ruby setup.rb config
ruby setup.rb setup
ruby setup.rb install

Install MPlayer:

1
2

cd /usr/src/MPlayer-1.0rc2
./configure && make && make install

Install ffMPEG:

1
2
3
4
5
6
7
8

cd /usr/src/ffmpeg/
mkdir tmp
chmod 777 tmp
export TMPDIR=./tmp
./configure --enable-libmp3lame --enable-libvorbis --disable-mmx --enable-shared
echo '#define HAVE_LRINTF 1' >> config.h
make && make install
export TMPDIR=/tmp

Finalize the codec setup:

1
2
3
4
5

ln -s /usr/local/lib/libavformat.so.50 /usr/lib/libavformat.so.50
ln -s /usr/local/lib/libavcodec.so.51 /usr/lib/libavcodec.so.51
ln -s /usr/local/lib/libavutil.so.49 /usr/lib/libavutil.so.49
ln -s /usr/local/lib/libmp3lame.so.0 /usr/lib/libmp3lame.so.0
ln -s /usr/local/lib/libavformat.so.51 /usr/lib/libavformat.so.51

You may get an error about a library path not being found, if so, run:

1

export LD_LIBRARY_PATH=/usr/local/lib

Install FFMPEG-PHP:

1
2
3
4
5

cd /usr/src/ffmpeg-php-0.5.0/
phpize
./configure && make && make install
ln -s /usr/local/bin/ffmpeg /usr/bin/ffmpeg
ln -s /usr/local/bin/mplayer /usr/bin/mplayer

Add extension to php.ini (find the correct php.ini file):

1
2
3

[ffmpeg]
extension_dir=/usr/local/lib/php/extensions/no-debug-non-zts-20060613/
extension=ffmpeg.so

Restart Apache and check that the module is loaded in PHP:

1

/etc/init.d/httpd restart

Test ffmpeg from command line and if you get this errors:

1

ffmpeg: error while loading shared libraries: libavformat.so.51:...

execute: echo "/usr/local/lib" >>/etc/ld.so.conf and reload library cache with ldconfig -v

Verify ffmpeg installation:

1

php -r 'phpinfo();' | grep ffmpeg

If you get the following results then FFMPEG and all it’s components are installed correctly:

1
2
3
4
5
6

ffmpeg
ffmpeg support (ffmpeg-php) => enabled
ffmpeg-php version => 0.5.3.1
ffmpeg-php gd support  => enabled
ffmpeg.allow_persistent => 0 => 0
ffmpeg.show_warnings => 0 => 0

Considering that you have an existing php installation compiled from sources, it is very easy to add afterwards new php modules, outside the php binary. This can be achieved without any interruption of the web services, and requires just the following steps:

1. Ensure you have the mcrypt libraries and development tools.

For example on a rhel system you will have to install: libmcrypt and libmcrypt-devel. If you can’t find those on your repos, dag has rpms for all rhel releases.

2. Compile the mcrypt module:

Assuming you have your php sources inside /usr/local/php, you have to: cd /usr/local/php cd ext/mcrypt phpize ./configure make make install just as on any 3rd party php module.

3. Finally activate the module in your php.ini

by adding the line: extension=mcrypt.so you will also want to restart your apache service in order to see the new php module installed.