Linux Tips & Tricks

First, you need to create a password file. Exactly how you do this will vary depending on what authentication provider you have chosen. More on that later. To start with, we’ll use a text password file.

This file should be placed somewhere not accessible from the web. This is so that folks cannot download the password file. For example, if your documents are served out of /usr/local/apache/htdocs you might want to put the password file(s) in /usr/local/apache/passwd.

To create the file, use the htpasswd utility that came with Apache. This will be located in the bin directory of wherever you installed Apache. If you have installed Apache from a third-party package, it may be in your execution path.

To create the file, type:

htpasswd -c /usr/local/apache/passwd/passwords rbowen

htpasswd will ask you for the password, and then ask you to type it again to confirm it:

# htpasswd -c /usr/local/apache/passwd/passwords rbowen
New password: mypassword
Re-type new password: mypassword
Adding password for user rbowen

If htpasswd is not in your path, of course you’ll have to type the full path to the file to get it to run. With a default installation, it’s located at /usr/local/apache2/bin/htpasswd

Next, you’ll need to configure the server to request a password and tell the server which users are allowed access. You can do this either by editing the httpd.conf file or using an .htaccess file. For example, if you wish to protect the directory /usr/local/apache/htdocs/secret, you can use the following directives, either placed in the file /usr/local/apache/htdocs/secret/.htaccess, or placed in httpd.conf inside a section.

AuthType Basic
AuthName “Restricted Files”
# (Following line optional)
AuthBasicProvider file
AuthUserFile /usr/local/apache/passwd/passwords
Require user rbowen

Let’s examine each of those directives individually. The AuthType directive selects that method that is used to authenticate the user. The most common method is Basic, and this is the method implemented by mod_auth_basic. It is important to be aware, however, that Basic authentication sends the password from the client to the server unencrypted. This method should therefore not be used for highly sensitive data, unless accompanied by mod_ssl. Apache supports one other authentication method: AuthType Digest. This method is implemented by mod_auth_digest and is much more secure. Most recent browsers support Digest authentication.

The AuthName directive sets the Realm to be used in the authentication. The realm serves two major functions. First, the client often presents this information to the user as part of the password dialog box. Second, it is used by the client to determine what password to send for a given authenticated area.

So, for example, once a client has authenticated in the “Restricted Files” area, it will automatically retry the same password for any area on the same server that is marked with the “Restricted Files” Realm. Therefore, you can prevent a user from being prompted more than once for a password by letting multiple restricted areas share the same realm. Of course, for security reasons, the client will always need to ask again for the password whenever the hostname of the server changes.

The AuthBasicProvider is, in this case, optional, since file is the default value for this directive. You’ll need to use this directive if you are choosing a different source for authentication, such as mod_authn_dbm or mod_authn_dbd.

The AuthUserFile directive sets the path to the password file that we just created with htpasswd. If you have a large number of users, it can be quite slow to search through a plain text file to authenticate the user on each request. Apache also has the ability to store user information in fast database files. The mod_authn_dbm module provides the AuthDBMUserFile directive. These files can be created and manipulated with the dbmmanage program. Many other types of authentication options are available from third party modules in the Apache Modules Database. For more information, visit our www.afgsolutions.com web site

Finally, the Require directive provides the authorization part of the process by setting the user that is allowed to access this region of the server. In the next section, we discuss various ways to use the Require directive.

Letting more than one person in

The directives above only let one person (specifically someone with a username of rbowen) into the directory. In most cases, you’ll want to let more than one person in. This is where the AuthGroupFile comes in.

If you want to let more than one person in, you’ll need to create a group file that associates group names with a list of users in that group. The format of this file is pretty simple, and you can create it with your favorite editor. The contents of the file will look like this:

GroupName: rbowen dpitts sungo rshersey

That’s just a list of the members of the group in a long line separated by spaces.

To add a user to your already existing password file, type:

htpasswd /usr/local/apache/passwd/passwords dpitts

You’ll get the same response as before, but it will be appended to the existing file, rather than creating a new file. (It’s the -c that makes it create a new password file).

Now, you need to modify your .htaccess file to look like the following:

AuthType Basic
AuthName “By Invitation Only”
# Optional line:
AuthBasicProvider file
AuthUserFile /usr/local/apache/passwd/passwords
AuthGroupFile /usr/local/apache/passwd/groups
Require group GroupName

Now, anyone that is listed in the group GroupName, and has an entry in the password file, will be let in, if they type the correct password.

There’s another way to let multiple users in that is less specific. Rather than creating a group file, you can just use the following directive:

Require valid-user

Using that rather than the Require user rbowen line will allow anyone in that is listed in the password file, and who correctly enters their password. You can even emulate the group behavior here, by just keeping a separate password file for each group. The advantage of this approach is that Apache only has to check one file, rather than two. The disadvantage is that you have to maintain a bunch of password files, and remember to reference the right one in the AuthUserFile directive.

Possible problems

Because of the way that Basic authentication is specified, your username and password must be verified every time you request a document from the server. This is even if you’re reloading the same page, and for every image on the page (if they come from a protected directory). As you can imagine, this slows things down a little. The amount that it slows things down is proportional to the size of the password file, because it has to open up that file, and go down the list of users until it gets to your name. And it has to do this every time a page is loaded. There are other solutions and we will write about them extensively later.

A consequence of this is that there’s a practical limit to how many users you can put in one password file. This limit will vary depending on the performance of your particular server machine, but you can expect to see slowdowns once you get above a few hundred entries, and may wish to consider a different authentication method at that time.

Although there are many shells available for Fedora, RedHat and its clones, such as tcsh, ksh, sh and the like, up until now bash still remains the default shell of choice.  This does not come as a surprise because bash (which stands for Bourne Again Shell) has numerous lightning fast built-in commands that can manipulate and explore system on large and limited scales.

Many times I observed, that Linux sysadmin beginners remember remember well how to move among the commands by pressing up and down arrows. They also remember how to use Tab in the command line for completion. Yet, somehow they forget that a great bash command history will pull out the complete history list of commands that they used.

Naturally, getting to see the whole list of commands might not come handy. It is much better, for example,  to go back to the last 10 commands and type in the terminal history 10. This will do the trick.

There are so many other built-in bash history commands that it will take hefty hundred pages to describe them all. I just want to mention another useful history manipulation command that is called simply fc.  This one is very valuable becaue it allow you to open vi editor and run the command after editing it to your liking. For example, fc 356 will allow you to edit this command in vi and then immediately invoke it for you.

Lots of Linux and Unix power users know how to share remote directories with Samba or NFS.  Unfortunately,  more and more malicious hackers get access to servers through these ways of sharing. For example, if somebody got one of Trojan horse’s access to your Windows machine and you access your server through it, there is a fat chance that the remote directories may be infiltrated to.

There is a solution to that. You can share your remote directories through the SSH file system.  You just need to make sure that your remote server is running SSH (which it usually does) and that it is accessible to your user account on a client machine.

If all this is true, you will need to install with your yum software that is called fuse-sshfs.  Then, naturally, you will need to create a mount point - a directory on your client machine for mounting data from a remote server to your local directory.

When you are done with these simple tasks,  you can start mounting the remote directory like that:

sshfs alex@10.0.0.13:/var/yourremotefolder /mnt/yourlocalfolder

As soon as you finish your work and want to unmount the remote directory, you will need to use the following fusermount command:

fusermount -u /mnt/yourlocalfolder

This solutions will be much safer for communications between Linux machines due to the nature of SSH encryption. Try it, I guarantee that you will like it.

I just read that Ubuntu 8.10 version is out as Server and Desktop editions that are available for download from Ubuntu.com website.  It is full of great features and new additions to the existing applications. After I heard the news I decided, that I would definitely test new Ubuntu Server in my web analytics company on one of my spare boxes.

For the last year I was dutifully installing each and every version of Ubuntu. Here is the reason why.  In the sea of Linux editions I always favored Debian on which Ubuntu was built. It seemed to me that Debian stability and Ubuntu implementation combined would produce a great competitor to RedHat and Fedora.

However, although I liked the way Ubuntu looks and easy updates for the last versions,  I could never get used to sudo commands (although it is wrong) and the added complexity of each version.  I had my reasons why I could never convince myself running Ubuntu as a production server.

I was always bothered that if something went wrong I would not be capable to find a quick fix or find good server support from forums. Although, people always praised Desktop edition (and I like it too), we are Linux system administrators and just use Ubuntu desktop for fun and testing only.

New 8.10 version ships with a Virtual Machine Builder, Open JDK and Apache Tomcat. This makes me conclude that emphasis on Java will be the priority for Server version. Well, I am not that well-versed in Java to write my own Java applications and use them in production environment.  I guess, these features will be more appealing to bigger corporations that keep Java programmers on their payroll.

New version also includes ClamAV and SpamAssassin which we’ve been using for ages on RedHat servers, so nothing is new here. Improved RAID support is OK, but I’d still recommend a decent hardware instead using just SATA software RAID controllers.

 When I read about new security features of Ubuntu server, I felt many new questions were popping up in my mind. Encrypted private directory for users? What if something goes wrong? How will I be able to quickly fix this problem while multiple users will be complaining that they can not log in either locally or through ssh?

Praised simplicity of Uncomplicated Firewall does not thrill me either. It really is not a step up from ipchains and other open source firewalls. And last but not the least is the Landscape Client that allows automated monitoring of the system. Well, did not we automate reporting a while ago?

Overall, I do want to use a shiny new Linux server with great features.  So, I do apologize for being a sceptic. But throughout the years, I learned that added complexity adds costs and headaches for Linux sysadmins. This means that we will have to go on with the prolonged testing that can take as long as 90 days. I promise to keep you posted on the results.

During updates your old Linux kernels pile up.  Then one day when you try to do your next updates, an interesting alert comes up from the server.  Basically, it tells you that your boot partition is full and can not fit any more kernels.  Sounds easy, right? It will be easy for those of you who is pretty crafty with terminal and debugging.  We have seasoned sysadmins in our web analytics company, but, naturally, there are junior guys who are still in constant learning process.

Unfortunately, those of them who are used to web interface, like Webmin, will delete their old kernels but won’t be able to proceed with the installation of a new Linux kernel. Interestingly enough, great Webmin just does not show all installed kernels. Some kind of a bug, I guess.

Well, your solution is to open the terminal and find out how many kernels are there anyways. So you need to type up:
rpm -q kernel | sort

Then you need to find out which kernel is the default one on your machine. So invoke the following command on your terminal:

uname -r

Then while whistling some funny melody, start uninstalling one by one all useless kernels. Here is the command:

rpm -e kernel(and its version)

Don’t forget to reboot.  The alert would not bother you anymore.

Note: Please, be careful with uninstallation. Don’t erase all kernels, leave at least a couple of them in case a new update is gonna cause you problems.  In that case, you will just switch back to the old kernel and wait for another kernel update.

What I love about our global Linux community - is that everybody tries to help the other. Maybe, it is changing, but I never heard anything like “pay me and I will tell you why this thing is not working”.  Of course, not everybody is savvy, some folks just give you plain wrong advice, so watch it, guys.

Yet, one may see, who knows his subject really well, when you start meeting the guy and try to do what he says and it, basically, works. In the previous entry I was talking about CentOS and how good it is.  Now, there is this guy whose nickname is “falko”. I don’t know where he is from. For all I know, he is from Europe, possibly from Berlin (but I could be wrong).

For several years he helps Linux community with Howto tutorials. He has the whole series that he calls “perfect server”, where he tells how to set up a LAMP server with DNS, E-mail and FTP and put on top of it open source web hosting panel ISPConfig. I can’t agree 100 % with what he recommends and this is understandable. First of all, I don’t use ISPConfig, I avoid ProFTPd and  I am not in web hosting business, I work for web analytics company.  Yet, I must admit, that falko is pretty consistent and with a little tweaking,  you can adjust his recommendations to your needs.

So, I appreciate this guy’s help because he has been writing his stuff about perfect CentOs, Fedora and RedHat Linux server installation for several years. As soon as a new version appears, I bet there will be  new falko’s tutorials with pictures and detailed explanations. I also like that in each tutorial he tells how to set up Postfix with TLS, and that part always works, if you follow him line by line.

To make a long story short,  I recommend you to try his latest stuff, related to CentOS installation for web hosting, and if you like it you can read more of his words of wisdom spread all over  HowToForge web site.

I still like RedHat servers and Fedora, what can I say? It is like an old heartache that never really goes away. I used to install RedHat Linux on multiple boxes when it was really free, if you know what I mean. Fedora, naturally, was another story. Due to its experimental nature, it was kind of risky to set it up as a production server - too much headache if something goes wrong. But I know a lot of system administrators braver than me, who, actually, run Fedora versions as production servers.

Well, I don’t know about you, guys, but I can’t afford to pay thousands of bucks for RedHat server “support”. That is why, it was such  a relieve for me when CentOS showed up and  saved the day. For those who does not know (is there anyone, I wonder?),  CentOS is a clone of RedHat server and a damn good one.  Its name has nothing to do with cents. This is just an abbreviation for Community Enterprise Operating System.

When I downloaded for free and installed my first CentOS I could not believe my eyes - it turned out as stable as its expensive daddy RedHat.  And if fact, that is what this company aims to - to be 100% binary compatible, as RedHat. You know that when a company has great product, in most cases this simple fact guarantees company’s longevity. I don’t know, if the guys there make any decent money. All I can say is that CentOS recently turned its 4th year of existence - this proves something, doesn’t it?

CentOS is pretty consistent, they produce a new version as soon as RedHat comes out with a new server.  So, I would say, it is, basically, the same as RedHat, but better, because it is free. And a number of web hosting companies are using CentOS for the same reasons.  And my web analytics company is no exception. So, why don’t you go and try CentOS too?

We all had problems with Linux and sometimes had to spend hours searching for solutions, googling and surfing message boards. Would not it be nice if the whole troubleshooting guide will be in one place?

There are many sites that try to create that kind of all-encompassing tutorial for troubleshooting with different level of success.  I can recommend one of them that stands aside from the others. This Wiki deals with software and debugging mostly, but people of various Linux/Unix skills might fine them intriguing. So, read on!

In my Web Analytics company, I occasionally had problems with Linux servers, because they tended to cache a lot of stuff. Especially, this problem occured, when I was manually copied a lot of files or was doing a tar backups for one of our clients. Well, there is an easy  solutions for this issue now

Kernels 2.6.16 and newer provide a mechanism to have the kernel drop the page cache and/or inode and dentry caches on command, which can help free up a lot of memory. Now you can throw away that script that allocated a ton of memory just to get rid of the cache…

To use /proc/sys/vm/drop_caches, just echo a number to it.

To free pagecache:

# echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

# echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

echo 3 > /proc/sys/vm/drop_caches

As this is a non-destructive operation and dirty objects are not freeable, the user should run “sync” command first!

I can’t express how happy I was when I built my first Samba server. I used my old box that was no good for running Windows file-and-print server, did not have enough memory or storage for that “perfect” Microsoft configuration.

This was a while ago. Those who did not have their own Samba, had to cough up a chunk of money for Windows “solutions”.

Anyways, when I saw that server icon from my Windows workstation, I was walking on heaven… After a while, I thought and created networking environment for all my printers. They started running off my mentioned above Samba server, unclogging my not -so-fast home LAN.

When hard drives became much bigger and printers more sophisticated, having Samba servers made even more sense. Anything, just anything that was not worth while keeping on our workstations went to Samba servers. Each person had his own share, where he could work and in the end of the day save all his stuff. Windows workstations were running faster and no information was getting lost or slipping through the cracks.

There is an unthinkable number of configurations that you could for your home LAN, office LAN, medium-size or big corporation. The settings are are easy, as a rule, if you know what you are doing. If you don’t, well, go to the main Samba site, click on Documentation link and knock yourself out. I swear, you won’t regret it.