e r i k i m h d o t c o m

So your IMAP mail is failing. Your e-mail client disconnects with an error related to bad authentication or simply “connection closed by remote server”. Check your mail logs, and you find:

May 25 17:52:43 vps imapd: Failed to create cache file: maildirwatch (someone@somewhere.com)
May 25 17:52:43 vps imapd: Error: Input/output error
May 25 17:52:43 vps imapd: Check for proper operation and configuration
May 25 17:52:43 vps imapd: of the File Access Monitor daemon (famd).

Found this obscure error in the system logs for IMAP. The server does not run “famd”. Make any sense? Not really. Though verbose, the output is indicative of nothing related to famd, I/O, or the cache file.

I first restarted the courier-auth daemon, which alleviated the failed login issue (seen as authentication failed or connection closed on most IMAP clients).

Second, edit the file:

/usr/lib/courier-imap/etc/imapd

and make sure IMAP_USELOCKS and IMAP_ENHANCEDIDLE are both set to 0, after that restart courier-imap:

/etc/init.d/courier-imap restart

Done, and done.

Share on Facebook

I came across this strange issue from a Tier II escalation today. A Virtuozzo based virtual server had a problem with the apache web server refusing to start:

# service httpd restart
[Sat May 15 16:41:13 2010] [warn] NameVirtualHost x.x.x.x:80 has no VirtualHosts
httpd not running, trying to start

Lets see if it’s actually started:

# service httpd status
Looking up localhost
Making HTTP connection to localhost
Alert!: Unable to connect to remote host.

Lets figure out what’s going on… Here’s some information on the kernel and OS:

-bash-3.00# uname -a
Linux xxxx 2.6.9-023stab046.2-enterprise #1 SMP Mon Dec 10 15:22:33 MSK 2007 i686 i686 i386 GNU/Linux

Normal troubleshooting ensued from there, and you can use this as a basis for determining what’s actually going on with your own server.

1. ALWAYS check the Apache error logs

Take a look at the error logs (usually “/usr/local/apache/logs”) and see if you can find what’s causing the problem.

In this case, the error_log gave me some valuable information and a place to start.

[Sat May 15 17:03:19 2010] [notice] suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec)
[Sat May 15 17:03:19 2010] [crit] (28)No space left on device: mod_rewrite: could not create rewrite_log_lock
Configuration Failed

On some server environments, you may also see a similar error that says:

[emerg] (28)No space left on device: Couldn’t create accept lock

Seems pretty obvious, yeah? Not so much….

2. Check available disk space.

-bash-3.00# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vzfs 77G 21G 57G 27% /

So we have plenty of disk space. Why can’t apache create the lockfile, then? Next step, (especially on virtual environments):

3. Check your available inodes

-bash-3.00# df -ih
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/vzfs 489K 167K 323K 34% /

**Scratches Head** So, the filesystem and available inodes are fine too. Now this gets interesting. The problem is that apache didn’t shut down properly, and it’s left myriads of semaphore-arrays left, owned by the apache-user (nobody).

A semaphore is a programming concept that is frequently used to solve multi-threading problems. Think of semaphores as bouncers at a nightclub. There are a dedicated number of people that are allowed in the club at once. If the club is full no one is allowed to enter, but as soon as one person leaves another person might enter.

To see if this is your problem, run:

ipcs -s | grep nobody

If you see a “wall” of these stragglers listed, your problem is solved. Removing these semaphores immediately should solve the problem and allow apache to start.

To do this, simply execute this command:

ipcs -s | grep nobody | perl -e 'while () { @a=split(/\s+/); print `ipcrm sem $a[1]`}'


You will see all of them being removed sequentially, and you can now go ahead and start up your apache service successfully.

PROTIP: Hitting the reset switch is NEVER the solution.

Share on Facebook

“Open Source Tripwire® software is a security and data integrity tool useful for monitoring and alerting on specific file change(s) on a range of systems. The project is based on code originally contributed by Tripwire, Inc. in 2000.”

A crude yet effective intrusion detection system such as Tripwire can alert systems administrators to possible intrusion attempts by periodically verifying the integrity of a server’s file systems. Systems intruders will often use trojan binaries for login, su, ps, and ls, etc. to cover their tracks and keep a low profile on the system. Under normal circumstances even astute systems administrators may not observe the intrusion because the trojan binaries mimic the system binaries so well.

This tool is a MUST have for all security conscious Linux System administrators. For step by step instructions on how to implement this on your server, you can [read entire article here].

Thanks to Linux Security, The central voice for Linux and Open Source security news for this very helpful information. Additionally, Yunliang Yu from Duke University has written a HOWTO on implementing this across an entire network. You can read his article here.

Share on Facebook

Yes folks, Ubuntu 10.04 is here. It’s suppose to be noticeably faster than 9.10, on any system. I’ve tested it on almost every device I have, from netbooks, to notebooks, to desktops, and high end servers. Two words: It’s awesome. Along with many new features, including a “Me Menu” and Ubuntu Software Center 2.0. Also included is a music store, “Ubuntu One Music Store”. It also marks the first time GIMP does not come preinstalled. Three versions are available, Desktop, Netbook, and LTS(Long Term Support). You may want to torrent the ISO, as Ubuntu’s servers have been stressed with the overwhelming demand over the last 48 hours!

Share on Facebook

Disclaimer: ALWAYS BACKUP YOUR DATA WHEN MAKING ANY CHANGES TO ANY FILE SYSTEMS. I will not be held responsible for any damage that you may incur as a result of following these instructions, and this should be used for informational purposes only, and as an outline of a process that worked for me on my particular hardware. KTHX.

First of all, make sure you have a fully updated system, regardless of your Ubuntu version.

Press Alt + F2 on your keyboard and type in

update-manager -d

The Update Manager will appear. Click on the Upgrade button. After the updates have completed, close and move on…

Now go to System->Partition Editor. This will show all the partition in your hard disk. Record down the filesystem ID of the partition that you want to convert to ext4.

Now, open a terminal and type the following:

sudo tune2fs -O extents,uninit_bg,dir_index /dev/XXXXXX

(Replace XXXX with the filesystem ID that you learned earlier)

sudo fsck -pf /dev/XXXXXX

Now, mount the drive:

sudo mount -t ext4 /dev/XXXXXX /mnt

You will now need to edit your system’s fstab file, which you can find in /mnt/etc/fstab (after mounting above) and change the ext3 entry to ext4.

Now you’re done, reinstall grub, reboot, and cross ya fingahz:

sudo grub-install /dev/XXXXXX
shutdown -r now

Share on Facebook

I’ve started testing Ubuntu 10.04 beta 1, the server edition on an older model Dell Poweredge, as well as the desktop edition on my Dell Inspiron laptop. So far, so good. The fresh install onto an ext4 file system was simple and efficient. For those of you who aren’t willing to “fresh” install it, I’d wait for the final release as the hacked together aptitude repository upgrades plastered all over the internet failed miserably.

Share on Facebook

Easyapache accepts custom flags in compiling, all of which can be added globally or using the following files supporting various environments:

* Apache 1.3.x – /var/cpanel/easy/apache/rawopts/Apache1
* Apache 2.0.x – /var/cpanel/easy/apache/rawopts/Apache2
* Apache 2.2.x – /var/cpanel/easy/apache/rawopts/Apache2_2
* All PHP 4.x versions – /var/cpanel/easy/apache/rawopts/all_php4
* All PHP 5.x versions – /var/cpanel/easy/apache/rawopts/all_php5
* Mod_suPHP – /var/cpanel/easy/apache/rawopts/all_suphp
* Specific PHP Version – /var/cpanel/easy/apache/rawopts/PHP-X.X.X

For typical PHP5 installations, you would simply add this to: /var/cpanel/easy/apache/rawopts/all_php5

Share on Facebook

Came across this rather annoying error when importing a rather large database into mySQL.

Here’s an easy fix:

Open mySQL console as root:

mysql> set global max_allowed_packet=1000000000;
mysql> set global net_buffer_length=1000000;

Then import your database:

# mysql db_name < sqldump.sql

Done.

Share on Facebook

There are instances where a full htaccess redirect rule is not the best solution, specifically to enforce only a single page to be encrypted. This may occur during PCI compliance scans which report an insecure login page. In this example, simply adding the following code to the header of the SSL-desired page will force it to revert to a secure connection:

(paste inside PHP brackets)

// MAKING SECURE PAGE LOAD

$secure_url = sprintf("%s%s%s","https://",$HTTP_HOST,$REQUEST_URI);
if(isset($_SERVER["HTTPS"])){
// do nothing
} else {
Header("Location: $secure_url");
}
// MADE SECURE PAGE LOAD

Share on Facebook

By default, bind will allow recursive queries for lookups on other domains that are not master zones on the name server.

This presents some PCI compliance issues and some informational vulnerabilities (allowing third parties to query the nameserver).

It is important to restrict who can perform DNS queries, in addition to what is allowed to be queried. If this DNS server is only meant to be recursively queried by internal users for third-party domains, then there is no reason to allow the general internet to also perform queries against it. If the server is meant only to act as a nameserver for specific domains, then recursive queries should be disabled as it is unnecessary for the server to resolve anything other than its own domains.

To disable recursive queries, add the following to the options section of named.conf:

allow-transfer {“none”;};
allow-recursion {“none”;};
recursion no;

Then restart the named service and dig at the name server to ensure the changes have taken effect.

Share on Facebook