This is a long article, but I hope you'll still find it interesting to read. Let me know if you want me to break down future long articles into multiple parts instead.
One of the most frequently asked questions around ZFS is: "How can I improve ZFS performance?".
This is not to say that ZFS performance would be bad. ZFS can be a very fast file system. ZFS is mostly self-tuning and the inherent nature of the algorithms behind ZFS help you reach better performance than most RAID-controllers and RAID-boxes - but without the expensive "controller" part.
Most of the ZFS performance problems that I see are rooted in incorrect assumptions about the hardware, or just unrealistic expectations of the laws of physics.
So let's look at ten ways to easily improve ZFS performance that everyone can implement without being a ZFS expert.
For ease of reading, here's a table of contents:
* The Basics of File System Performance
* Performance Expectations, Goals and Strategy
* #1: Add Enough RAM
* #2: Add More RAM
* #3: Boost Deduplication Performance With Even More RAM
* #4: Use SSDs to Improve Read Performance
* #5: Use SSDs to Improve Write Performance
* #6: Use Mirroring
* #7: Add More Disks
* #8: Leave Enough Free Space
* #9: Hire An Expert
* #10: Be An Evil Tuner - But Know What You Do
* Bonus: Some Miscellaneous Settings
* Your Turn
* Related Posts
But before we start with our performance tips, let's cover the basics:
The Basics of File System Performance
It's important to distinguish between the two basic types of file system operation:
* Reads, and
* Writes.
This may sound stupidly simple, but data travels very different paths through the ZFS I/O subsystem for reads vs. writes and this means there are different ways to improve read performance than there are to make writes faster.
Use zpool iostat or iostat(1M) and verify what read/write performance the system sees and whether it matches your observations and expectations.
Then there are two kinds of file system performance:
* Bandwidth: Measured in MB/s (or GB/s if you're lucky), telling you how much overall data passes through the system over time.
* IOPS: The number of IO operations that are carried out per second.
Again, these different ways of looking at performance can be optimized by different means, you just need to know into which category your particular problem falls.
There are also two patterns of read/write performance:
* Sequential: Predictable, one block after the other, lined up like pearls on a string.
* Random: Unpredictable, unordered, difficult to grasp.
The good news here is that ZFS automatically turns random writes into sequential writes through the magic of copy-on-write. One less class of performance problems to take care of.
And finally, for write I/Os, you should know about the difference between:
* Synchronous Writes: Writes that are only complete after they have been successfully written to stable storage. In ZFS, they're implemented through the ZFS Intent Log, or ZIL. These are most often found in file and database servers and these kinds of writes are very sensitive to latency or IOPS performance.
* Asynchronous Writes: Write operations that may return after being cached in RAM, before they are committed to disk. Performance is easy to get here, at the expense of reliability: If the power fails before the buffer is written to disk, data can be lost.
Performance Expectations, Goals and Strategy
We're almost there, but before we get to the actual performance tips, we need to discuss a few methodical things:
* Set realistic expectations: ZFS is great, yes. But you need to observe the laws of physics. A disk with 10000 rpm can't deliver more than 166 random IOPS because 10000 revolutions divided by 60 seconds (per minute) means the head can only position itself above a random block 166 times per second. Any more than that and your data is not really random. That's just how the numbers play out.
Similarly, RAID-Z means that you'll only get the IOPS performance of a single disk per RAID-Z group, because each filesystem IO will be mapped to all the disks in a RAID-Z group in parallel.
Make sure you know what the limits of your storage are and what performance you can realistically expect, when analyzing your performance and setting performance goals. By the way...
* Define performance goals: What exactly is "too slow"? What performance would be acceptable? Where are you now, and where do you want to be?
Performance goals are important to set, because they tell you when you're done. There are always ways to improve performance, but there's no use in improving performance at all costs. Know when you're done, then celebrate!
* Be systematic: It happens so many times: We try this, then we try that, we measure with cp(1) even though our app is actually a database, then we tweak here and there, and before we know it, we realize: We really know nothing.
Being systematic means defining how to measure the performance we want, establishing the status quo, in a way that is directly related to the actual application we're interested in, then sticking to the same performance measurement method through the whole performance analysis and optimization process.
Otherwise, things become confusing, we lose sight of where we are and we won't be able to tell if we reached our goal or not.
Now that we have an understanding of the kind of performance we want, we know what we can expect from today's hardware, we defined some realistic goals and have a systematic approach to performance optimization, let's begin:
#1: Add Enough RAM
A small amount of data on your disks is spent for storing ZFS metadata. This is the data that ZFS needs, so it knows where your actual data is. In a way, this is the roadmap that ZFS needs to find its way through your disks and the data structures there.
If your server doesn't have enough RAM to store metadata, then it will need to issue extra metadata read IOs for every data read IO to figure out where your data actually is on disk. This is slower than necessary, and you really want to avoid that. If you're really short on RAM, this could have a massive impact!
How much RAM do you need? As a rough rule of thumb, divide the size of your total storage by 1000, then add 1 GB so the OS has some extra RAM of its own to breathe. This means for every TB of data, you'll want at least 1GB of RAM for caching ZFS metadata, in addition to one GB for the OS to feel comfortable in.
Having enough RAM will benefit all of your reads, no matter if they're random or sequential, just because they'll be easier for ZFS to find on your disks, so make sure you have at least n/1000 + 1 GB of RAM, where n is the number of GB in your storage pool.
#2: Add More RAM
ZFS uses every piece of RAM it finds to cache data. It has a very sophisticated caching algorithm that tries to cache both most frequently used data, and most recently used data, adapting their balance while it's used. ZFS also has some advanced prefetching abilities that can greatly improve performance for different kinds of sequential reads.
All of this works the better the more RAM you give to ZFS. But when do you know if more RAM will give you breakthrough performance, or just a small improvement?
This is where your working set comes in.
Your working set is the part of your data that is used most often: Your top products/websites/customers in an e-commerce database. Your clients with the biggest traffic in your hosting environment. Your most popular files etc.
If your working set fits into RAM, the utter majority of reads can be serviced from RAM most of the time, without having to create any IOs to slow-spinning disks.
Try to figure out what the most popular subset of your data is, then add enough RAM to your ZFS server to help it live there. This will give you the biggest read performance boost.
If you want something more automated, Ben Rockwood has written a great tool called arc_summary (ARC is the ZFS Adaptive Replacement Cache). The two "Ghost" values tell you exactly how much more memory would have helped you to handle the load that your server has seen in the past.
If you want to influence the balance between user data and metadata in the ZFS ARC cache, check out the primarycache filesystem property that you can set using the zfs(1M) command. For RAM-starved servers with a lot of random reads, it may make sense to restrict the precious RAM cache to metadata and use an L2ARC, explained in tip #4 below.
#3: Boost Deduplication Performance With Even More RAM
In an earlier article, I wrote about the basics of ZFS Deduplication. If you plan to use it, keep in mind that ZFS will assemble a table of all the blocks stored in your filesystem and their checksums, so it can determine whether a specific block has been already written and can thus safely marked as a duplicate.
Deduplication will save you space and it can also add to your performance because it saves you unnecessary read and write IOPS. But the cost of this is the need to keep the dedup table as handy as possible, ideally in RAM.
How big is the ZFS dedup table? Richard Elling pointed out in a recent mailinglist post that a ZFS dedup table entry uses about 250 Bytes per data block. Assuming an average block size of 8K, a TB of user data would need about 32GB of RAM if you want to be real fast. If your data tends to be spread over large files, you'll have a bigger average blocksize, say, 64K, and then you'd only need about 4GB of RAM for the dedup table.
If you don't have that amount of RAM, there's no need to despair, there's always the possibility to...
#4: Use SSDs to Improve Read Performance
If you can't add any more RAM to your server (or if your purchasing department won't allow you), the next best way to increase read performance is to add solid state disks (aka flash memory) as a level 2 ARC cache (L2ARC) to your system.
You can easily configure them with the zpool(1M) command, read the "Cache devices" section of its man-page.
SSDs can deliver two orders of magnitude better IOPS than traditional harddisks, and they're much cheaper on a per-GB basis than RAM.
They form an excellent layer of cache between the ZFS RAM-based ARC and the actual disk storage.
You don't need to observe any reliability requirements when configuring L2ARC devices: If they fail, no data is lost because it can always be retrieved from disk.
This means that L2ARC devices can be cheap, but before you start putting USB sticks into your server, you should make sure they deliver a good performance benefit over your rotating disks :).
SSDs come in various sizes: From drop-in-replacements for existing SATA disks in the range of 32GB to the Oracle Sun F20 PCI card with 96GB of flash and built-in SAS controllers (which is one of the secrets behind Oracle Exadata V2's breakthrough performance), to the mighty fast Oracle Sun F5100 flash array (which is the secret behind Oracle's current TPC-C and other world records) with a whopping 1.96TB of pure flash memory and over a million IOPS. Nice!
And since the dedup table is stored in the ZFS ARC and consequently spills off into the L2ARC if available, using SSDs as cache devices will also benefit deduplication performance.
#5: Use SSDs to Improve Write Performance
Most write performance problems are related to synchronous writes. These are mostly found in file servers and database servers.
With synchronous writes, ZFS needs to wait until each particular IO is written to stable storage, and if that's your disk, then it'll need to wait until the rotating rust has spun into the right place, the harddisk's arm moved to the right position, and finally, until the block has been written. This is mechanical, it's latency-bound, it's slow.
See Roch's excellent article on ZFS NFS performance for a more detailed discussion on this.
SSDs can change the whole game for synchronous writes because they have 100x better latency: No moving parts, no waiting, instant writes, instant performance.
So if you're suffering from a high load in synchronous writes, add SSDs as ZFS log devices (aka ZIL, Logzillas) and watch your synchronous writes fly. Check out the zpool(1M) man page under the "Intent Log" section for more details.
Make sure you mirror your ZIL devices: They are there to guarantee the POSIX requirement for "stable storage" so they must function reliably, otherwise data may be lost on power or system failure.
Also, make sure you use high quality SLC Flash Memory devices, because they can give you reliable write transactions. Cheaper MLC cells can damage existing data if the power fails during write operations, something you really don't want.
#6: Use Mirroring
Many people configure their storage for maximum capacity. They just look at how many TB they can get out of their system. After all, storage is expensive, isn't it?
Wrong. Storage capacity is cheap. Every 18 months or so, the same disk only costs half as much, or you can buy double the capacity for the same price, depending on how you view it.
But storage performance can be precious. So why squeeze the last GB out of your storage if capacity is cheap anyway? Wouldn't it make more sense to trade in capacity for speed?
This is what mirroring disks offer as opposed to RAID-Z or RAID-Z2:
* RAID-Z(2) groups several disks into a RAID group, called vdevs. This means that every I/O operation at the file system level is going to be translated into a parallel group of I/O operations to all of the disks in the same vdev.
The result: Each RAID group can only deliver the IOPS performance of a single disk, because the transaction always has to wait until all of the disks in the same vdev are finished.
This is both true for reads and for writes: The whole pool can only deliver as many IOPS as the total number of striped vdevs times the IOPS of a single disk.
There are cases where the total bandwidth of RAID-Z can take advantage of the aggregate performance of all drives in parallel, but if you're reading this, you're probably not seeing such a a case.
* Mirroring behaves differently: For writes, the rules are the same: Each mirrored pair of disks will deliver the write IOPS of a single disk, because each write transaction will need to wait until it has completed on both disks. But a mirrored pair of disks is a much smaller granularity than your typical RAID-Z set (with up to 10 disks per vdev). For 20 disks, this could be the difference between 10x the IOPS of a disk in the mirror case vs. only 2x the IOPS of a disk in a wide stripes RAID-Z2 scenario (8+2 disks per RAID-Z2 vdev). A 5x performance difference!
For reads, the difference is even bigger: ZFS will round-robin across all of the disks when reading from mirrors. This will give you 20x the IOPS of a single disk in a 20 disk scenario, but still only 2x if you use wide stripes of the 8+2 kind.
Of course, the numbers can change when using smaller RAID-Z stripes, but the basic rules are the same and the best performance is always achieved with mirroring.
For a more detailed discussion on this, I highly recommend Richard Elling's post on ZFS RAID recommendations: Space, performance and MTTDL.
Also, there's some more discussion on this in my earlier RAID-GREED-article.
Bottom line: If you want performance, use mirroring.
#7: Add More Disks
Our next tip was already buried inside tip #6: Add more disks. The more vdevs ZFS has to play with, the more shoulders it can place its load on and the faster your storage performance will become.
This works both for increasing IOPS and for increasing bandwidth, and it'll also add to your storage space, so there's nothing to lose by adding more disks to your pool.
But keep in mind that the performance benefit of adding more disks (and of using mirrors instead of RAID-Z(2)) only accelerates aggregate performance. The performance of every single I/O operation is still confined to that of a single disk's I/O performance.
So, adding more disks does not substitute for adding SSDs or RAM, but it'll certainly help aggregate IOPS and bandwidth for the cases where lots of concurrent IOPS and bigger overall bandwidth are needed.
#8 Leave Enough Free Space
Don't wait until your pool is full before adding new disks, though.
ZFS uses copy on write which means that it writes new data into free blocks, and only when the überblock has been updated, the new state becomes valid.
This is great for performance because it gives ZFS the opportunity to turn random writes into sequential writes - by choosing the right blocks out of the list of free blocks so they're nicely in order and thus can be written to quickly.
That is, when there are enough blocks.
Because if you don't have enough free blocks in your pool, ZFS will be limited in its choice, and that means it won't be able to choose enough blocks that are in order, and hence it won't be able to create an optimal set of sequential writes, which will impact write performance.
As a rule of thumb, don't let your pool become more full than about 80% of its capacity. Once it reaches that point, you should start adding more disks so ZFS has enough free blocks to choose from in sequential write order.
#9: Hire A ZFS Expert
There's a reason why this point comes up almost last: In the utter majority of all ZFS performance cases, one or more of #1-#8 above are almost always the solution.
And they're cheaper than hiring a ZFS performance expert who will likely tell you to add more RAM, or add SSDs or switch from RAID-Z to mirroring after looking at your configuration for a couple of minutes anyway!
But sometimes, a performance problem can be really tricky. You may think it's a storage performance problem, but instead your application may be suffering from an entirely different effect.
Or maybe there are some complex dependencies going on, or some other unusual interaction between CPUs, memory, networking, I/O and storage.
Or perhaps you're hitting a bug or some other strange phenomenon?
So, if all else fails and none of the above options seem to help, contact your favorite Oracle/Sun representative (or send me a mail) and ask for a performance workshop quote.
If your performance problem is really that hard, we want to know about it.
#10: Be An Evil Tuner - But Know What You Do
If you don't want to go for option #9 and if you know what you do, you can check out the ZFS Evil Tuning Guide.
There's a reason it's called "evil": ZFS is not supposed to be tuned. The default values are almost always the right values, and most of the time, changing them won't help, unless you really know what you're doing. So, handle with care.
Still, when people encounter a ZFS performance problem, they tend to Google "ZFS tuning", then they'll find the Evil Tuning Guide, then think that performance is just a matter of setting that magic variable in /etc/system.
This is simply not true.
Measuring performance in a standardized way, setting goals, then sticking to them helps. Adding RAM helps. Using SSDs helps. Thinking about the right number and RAID level of disks helps. Letting ZFS breathe helps.
But tuning kernel parameters is reserved for very special cases, and then you're probably much better off hiring an expert to help you do that correctly.
Bonus: Some Miscellaneous Settings
If you look through the zfs(1M) man page, you'll notice a few performance related properties you can set.
They're not general cures for all performance problems (otherwise they'd be set by default), but they can help in specific situations. Here are a few:
* atime: This property controls whether ZFS records the time of last access for reads. Switching this to off will save you extra write IOs when reading data. This can have a big impact if your application doesn't care about the time of last access for a file and if you have a lot of small files that need to be read frequently.
* checksum and compression can be double-edged swords: The stronger the checksum, the better your data is protected against corruption (and this is even more important when using dedup). But a stronger checksum method will incur some more load on the CPU for both reading and writing.
Similarly, using compression may save a lot of IOPS if the data can be compressed well, but may be in the way for data that isn't easily compressed. Again, compression costs some extra CPU time.
Keep an eye on CPU load while running tests and if you find that your CPU is under heavy load, you might need to tweak one of these.
* recordsize: Don't change this property unless your running a database in this filesystem. ZFS automatically figures out what the best blocksize is for your filesystems.
In case you're running a database (where the file may be big, but the access pattern is always in fixed-size chunks), setting this property to your database record size may help performance a lot.
* primarycache and secondarycache: We already introduced the primarycache property in tip #2 above. It controls whether your precious RAM cache should be used for metadata or for both metadata and user data. In cases where you have an SSD configured as a cache device and if you're using a large filesystem, it may help to set primarycache=metadata so the RAM is used for metadata only.
secondarycache does the same for cache devices, but it should be used to cache metadata only in cases where you have really big file systems and almost no real benefit from caching data.
* logbias: When executing synchronous writes, there's a tradeoff to be made: Do you want to wait a little, so you can accumulate more synchronous write requests to be written into the log at once, or do you want to service each individual synchronous write as fast as possible, at the expense of throughput?
This property lets you decide which side of the tradeoff you want to favor.
Your Turn
Sorry for the long article. I hope the table of contents at the beginning makes it more digestible, and I hope it's useful to you as a little checklist for ZFS performance planning and for dealing with ZFS performance problems.
Let me know if you want me to split up longer articles like these (though this one is really meant to remain together).
Now it's your turn: What is your experience with ZFS performance? What options from the above list did you implement for what kind of application/problem and what were your results? What helped and what didn't and what are your own ZFS performance secrets?
Share your ZFS performance expertise in the comments section and help others get the best performance out of ZFS!
Related Posts
* Seven Useful OpenSolaris ZFS Home Server Tips
* OpenSolaris ZFS Deduplication: Everything You Need to Know
* Home Server: RAID-GREED and Why Mirroring is Still Best
Chủ Nhật, 24 tháng 10, 2010
Thứ Tư, 20 tháng 10, 2010
[Solaris] Installing Cool Stack Apache, MySQL, PHP
Coolstack is Sun's perferred suite of precompiled Apache 2.2.3, MySQL 5, and PHP5 all ready to go in a bundle. Each package is compiled optimized for performance on each architecture, and is fully tested by Sun.
Preflight
Prerequisites
For Coolstack 1.2, you must first install the Coolstack runtime package:
CSKruntime_1.2_x86.pkg
Cool Stack 1.2
Cool Stack 1.1
To install on Joyent Accelerators all you have to do is execute a few simple steps. Download Coolstack from http://cooltools.sunsource.net/coolstack/.
Once you get the package in your Accelerator, decompress it and pkgadd it:
# bzip2 -d CSKamp_x86.pkg.bz2
# pkgadd -d ./CSKamp_x86.pkg
By default it installs to /opt/coolstack. If you go inside this directory you will see directories for each individual package:
# ls /opt/coolstack
apache2/ etc/ info/ man/ php5/ share/
bin/ include/ lib/ mysql_32bit/ sbin/
If you go in to each application's directory, it will have a README file. In this file it will tell you steps for setting up and also the compile options used for Sun Studio.
Apache
The Apache used is 2.2.3 compiled with prefork. Apache is all ready to go, all you have to do is start it.
# /opt/coolstack/apache2/bin/apachectl start
MySQL
The MySQL used is version 5.0.33 32-bit. If you need to work with larger databases (use more than 4gigs of ram), consider the MySQL 64-bit package that the CoolStack webpage provides.
To get MySQL started, you need to first copy over a my.cnf file to use. my-medium.cnf should work for most Accelerators, check my-small.cnf if you are on a smaller container and want to conserve more memory.
# cp /opt/coolstack/mysql_32bit/share/mysql/my-medium.cnf /opt/coolstack/mysql_32bit/my.cnf
Also copy over the mysql.server start and stop script:
# cp /opt/coolstack/mysql_32bit/share/mysql/mysql.server /opt/coolstack/mysql_32bit
Create the system tables and give permissions to the directories:
# /opt/coolstack/mysql_32bit/bin/mysql_install_db
# chown -R mysql:mysql /opt/coolstack/mysql_32bit
And start the server:
# /opt/coolstack/mysql_32bit/bin/mysql.server start
Starting MySQL
SUCCESS!
Preflight
Prerequisites
For Coolstack 1.2, you must first install the Coolstack runtime package:
CSKruntime_1.2_x86.pkg
Cool Stack 1.2
Cool Stack 1.1
To install on Joyent Accelerators all you have to do is execute a few simple steps. Download Coolstack from http://cooltools.sunsource.net/coolstack/.
Once you get the package in your Accelerator, decompress it and pkgadd it:
# bzip2 -d CSKamp_x86.pkg.bz2
# pkgadd -d ./CSKamp_x86.pkg
By default it installs to /opt/coolstack. If you go inside this directory you will see directories for each individual package:
# ls /opt/coolstack
apache2/ etc/ info/ man/ php5/ share/
bin/ include/ lib/ mysql_32bit/ sbin/
If you go in to each application's directory, it will have a README file. In this file it will tell you steps for setting up and also the compile options used for Sun Studio.
Apache
The Apache used is 2.2.3 compiled with prefork. Apache is all ready to go, all you have to do is start it.
# /opt/coolstack/apache2/bin/apachectl start
MySQL
The MySQL used is version 5.0.33 32-bit. If you need to work with larger databases (use more than 4gigs of ram), consider the MySQL 64-bit package that the CoolStack webpage provides.
To get MySQL started, you need to first copy over a my.cnf file to use. my-medium.cnf should work for most Accelerators, check my-small.cnf if you are on a smaller container and want to conserve more memory.
# cp /opt/coolstack/mysql_32bit/share/mysql/my-medium.cnf /opt/coolstack/mysql_32bit/my.cnf
Also copy over the mysql.server start and stop script:
# cp /opt/coolstack/mysql_32bit/share/mysql/mysql.server /opt/coolstack/mysql_32bit
Create the system tables and give permissions to the directories:
# /opt/coolstack/mysql_32bit/bin/mysql_install_db
# chown -R mysql:mysql /opt/coolstack/mysql_32bit
And start the server:
# /opt/coolstack/mysql_32bit/bin/mysql.server start
Starting MySQL
SUCCESS!
Thứ Năm, 14 tháng 10, 2010
Configuring Sendmail to Relay Messages from Other Servers
Part 1. Configuring Sendmail on Solaris 10
Part 2. Configuring Sendmail to Masquerade Your Messages
Part 3. Configuring Sendmail to Relay Messages to Another Server
Part 4. Configuring Sendmail to Relay Messages from Other Servers
Introduction
In the previous post you've learnt how to configure Sendmail to relay messages to another server. Now, such a server should be probably be configured to accept incoming messages to relay from other servers. Solaris 10 Sendmail default configuration does not allow message relay and proper configuration must be applied to Sendmail.
Configuring Relay for Hosts and Domains
The quickest way to have Sendmail relay messages for other domains is by modifying the /etc/mail/relay-domains file. Sendmail will relay mail for every domain listed in that file. If you want your server to relay messages for domain a.com, b.com and c.com, just insert the corresponding lines into /etc/mail/relay-domains and restart your Senmail instance:
# cat /etc/mail/relay-domains
a.com
b.com
c.com
Configuring the Access Database
If you want to relay messages from specific hosts (as well as domains and networks) you can use the access database. The access database lists email addresses, network numbers and domain names and a rule. Available rules are:
* OK: Accept mail even if other rules in the running ruleset would reject it.
* RELAY: Accept mail addressed to the indicated domain or received from the indicated
domain for relaying.
* REJECT: Reject the sender or recipient with a general purpose message.
* DISCARD: Discard the message completely using the $#discard mailer.
* (A RFC-821 compliant error text): Return the error message.
If you want your Sendmail to relay mails for a domain or from some specific hosts, modify your /etc/mail/access accordingly:
your-domain RELAY
192.168.0 RELAY
another-domain RELAY
unwanted-host REJECT
Once done, you have to generate the access db with the following command:
# makemap hash /etc/mail/access.db < /etc/mail/access
Enabling the Access Database
To have your Sendmail use the access database, you must properly configure it adding the access_db feature to its configuration file:
# cat your-file.mc
[...snip...]
FEATURE(`access_db')
[...snip...]
Restart your Sendmail and enjoy!
A Word of Warning: DNS Configuration
Sendmail often requires that host name you use in your configuration files (such as the access database) are properly configured in your name server, both for lookup and reverse lookup. I hope this will spare you some headache while debugging.
Part 2. Configuring Sendmail to Masquerade Your Messages
Part 3. Configuring Sendmail to Relay Messages to Another Server
Part 4. Configuring Sendmail to Relay Messages from Other Servers
Introduction
In the previous post you've learnt how to configure Sendmail to relay messages to another server. Now, such a server should be probably be configured to accept incoming messages to relay from other servers. Solaris 10 Sendmail default configuration does not allow message relay and proper configuration must be applied to Sendmail.
Configuring Relay for Hosts and Domains
The quickest way to have Sendmail relay messages for other domains is by modifying the /etc/mail/relay-domains file. Sendmail will relay mail for every domain listed in that file. If you want your server to relay messages for domain a.com, b.com and c.com, just insert the corresponding lines into /etc/mail/relay-domains and restart your Senmail instance:
# cat /etc/mail/relay-domains
a.com
b.com
c.com
Configuring the Access Database
If you want to relay messages from specific hosts (as well as domains and networks) you can use the access database. The access database lists email addresses, network numbers and domain names and a rule. Available rules are:
* OK: Accept mail even if other rules in the running ruleset would reject it.
* RELAY: Accept mail addressed to the indicated domain or received from the indicated
domain for relaying.
* REJECT: Reject the sender or recipient with a general purpose message.
* DISCARD: Discard the message completely using the $#discard mailer.
* (A RFC-821 compliant error text): Return the error message.
If you want your Sendmail to relay mails for a domain or from some specific hosts, modify your /etc/mail/access accordingly:
your-domain RELAY
192.168.0 RELAY
another-domain RELAY
unwanted-host REJECT
Once done, you have to generate the access db with the following command:
# makemap hash /etc/mail/access.db < /etc/mail/access
Enabling the Access Database
To have your Sendmail use the access database, you must properly configure it adding the access_db feature to its configuration file:
# cat your-file.mc
[...snip...]
FEATURE(`access_db')
[...snip...]
Restart your Sendmail and enjoy!
A Word of Warning: DNS Configuration
Sendmail often requires that host name you use in your configuration files (such as the access database) are properly configured in your name server, both for lookup and reverse lookup. I hope this will spare you some headache while debugging.
Thứ Năm, 16 tháng 9, 2010
VSFTP command
ABOR,ACCT,ALLO,APPE,CDUP,CWD,DELE,EPRT,EPSV,FEAT,HELP,LIST,MDTM,MKD,MODE,NLST,NOOP,OPTS,PASS,PASV,PORT,PWD,QUIT,REIN,REST,RETR,RMD,RNFR,RNTO,SITE,SIZE,SMNT,STAT,STOR,STOU,STRU,SYST,TYPE,USER,XCUP,XCWD,XMKD,XPWD,XRMD
Reading FTP Logs in xferlog Format
For some reason I can never remember the xferlog format that is used by daemons such as Pure-FTP. Although xferlog is well documented, I can never seem to find the doc when I need it, and it's never bad to have information duplicated in many places!
Anyways, on with the description. Here is a sample log entry from my server (with access IPs and dirs changed):
I'll step through each item individually. The delimiter here is whitespace, so each new token represents a unique piece of data, with the exception of the date at the beginning.
Fri May 14 05:16:12 2010
Date/time stamp, nothing complicated.
0
Transfer time, in whole seconds (this transfer took less than a second, so zero).
::ffff:1.2.3.4
Remote host where the user connected from.
11974
Size of the transferred file (in bytes).
/home/user/public_html/index.php
Full path to the uploaded file.
a
Transfer type, a = ASCII (plain-text files), b = binary (everything else)
_
Action flag, C = compressed, U = uncompressed; T = tar'ed; _ = no action was taken.
i
Direction, i = incoming, o = outgoing, d = deleted.
r
Access mode, a = anonymous user, r = real (normal) user.
user
Local username authenticated with.
ftp
The service being invoked (almost always FTP).
0
Authentication method, 0 = none, 1 = RFC931 authetication.
*
User ID or * if not available (virtual user).
c
Completion status, c = completed, i = incomplete.
That's all there is to it!
Anyways, on with the description. Here is a sample log entry from my server (with access IPs and dirs changed):
Fri May 14 05:16:12 2010 0 ::ffff:1.2.3.4 11974 /home/user/public_html/index.php a _ i r user ftp 0 * c
I'll step through each item individually. The delimiter here is whitespace, so each new token represents a unique piece of data, with the exception of the date at the beginning.
Fri May 14 05:16:12 2010
Date/time stamp, nothing complicated.
0
Transfer time, in whole seconds (this transfer took less than a second, so zero).
::ffff:1.2.3.4
Remote host where the user connected from.
11974
Size of the transferred file (in bytes).
/home/user/public_html/index.php
Full path to the uploaded file.
a
Transfer type, a = ASCII (plain-text files), b = binary (everything else)
_
Action flag, C = compressed, U = uncompressed; T = tar'ed; _ = no action was taken.
i
Direction, i = incoming, o = outgoing, d = deleted.
r
Access mode, a = anonymous user, r = real (normal) user.
user
Local username authenticated with.
ftp
The service being invoked (almost always FTP).
0
Authentication method, 0 = none, 1 = RFC931 authetication.
*
User ID or * if not available (virtual user).
c
Completion status, c = completed, i = incomplete.
That's all there is to it!
Thứ Ba, 30 tháng 3, 2010
Step by step install innotop
Innotop is a very useful tool to monitor innodb information in real time. This tool is written by Baron Schwartz who is also an author of “High Performance MySQL, Second edition” book. [Side note: I highly recommend getting this book when it comes out (in June, 08?). Other authors include: Peter Zaitsev, Jeremy Zawodny, Arjen Lentz, Vadim Tkachenko and Derek J. Balling.] Quick summary of what innotop can monitor (from: http://innotop.sourceforge.net/): InnoDB transactions and internals, queries and processes, deadlocks, foreign key errors, replication status, system variables and status and much more.
Following are the instructions on how to install innotop on CentOS x64/Fedora/RHEL (Redhat enterprise). Most probably same instructions can be used on all flavors of Linux. If not, leave me a comment and I will research a solution for you. Let us start with downloading innotop. I used version 1.6.0 which is the latest at the time of writing.
wget http://internap.dl.sourceforge.net/sourceforge/innotop/innotop-1.6.0.tar.gz
Now let us go ahead and unzip and create the MakeFile to get it ready for install
tar zxpf innotop-1.6.0.tar.gz
cd innotop-1.6.0
perl Makefile.PL
At this point if you get the following output, you are good to continue:
Checking if your kit is complete...
Looks good
Writing Makefile for innotop
If you get something similar to following, you will need to take care of the prerequisites:
Looks good
Warning: prerequisite DBD::mysql 1 not found.
Warning: prerequisite DBI 1.13 not found.
Warning: prerequisite Term::ReadKey 2.1 not found.
Writing Makefile for innotop
Just because they are warnings does not mean you ignore them. So let us install those prerequisites. We will use perl’s cpan shell to get this installed (visit my post on how to install perl modules for more details). If it is your first time starting this up, you will have to answer some questions. Defaults will work fine in all cases.
perl -MCPAN -eshell
install Term::ReadKey
install DBI
install DBD::mysql
Note: you must install DBI before you can install DBD::mysql.
If you get an error like following when you are installing DBD::mysql:
Error: Can't load '/root/.cpan/build/DBD-mysql-4.007/blib/arch/auto/DBD/mysql/mysql.so' for module DBD::mysql: libmysqlclient.so.15: cannot open shared object file: No such file or directory at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230.
You will have to create a symlink to the object file in your lib64 (or lib if you are not using x64 version) folder:
ln -s /usr/local/mysql/lib/mysql/libmysqlclient.so.15 /lib64/
Once all prerequisites are done, type perl Makefile.PL and you should have no warnings. Continue the install:
make install
At this point you should have innotop installed on your system. Let us do some quick set up so you can start using innotop. We start with configuring your .my.cnf to include connection directives.
vi ~/.my.cnf
Add the following (edit to reflect your install) and save/exit
[mysql]
port = 3306
socket = /tmp/mysql.sock
Start up innotop by typing innotop at your shell prompt. First prompt will ask you to “Enter a name:”. I just put localhost since this will be used to connect locally. Next prompt asks you about DSN entry. I use: DBI:mysql:;mysql_read_default_group=mysql
This tells innotop to read .my.cnf file and use group [mysql] directives. Next prompt is optional (I just press enter). Next two prompts you enter information if you need to.
At this point your innotop installation / testing is complete. You can read man innotop to get more details on how to use innotop.
Following are the instructions on how to install innotop on CentOS x64/Fedora/RHEL (Redhat enterprise). Most probably same instructions can be used on all flavors of Linux. If not, leave me a comment and I will research a solution for you. Let us start with downloading innotop. I used version 1.6.0 which is the latest at the time of writing.
wget http://internap.dl.sourceforge.net/sourceforge/innotop/innotop-1.6.0.tar.gz
Now let us go ahead and unzip and create the MakeFile to get it ready for install
tar zxpf innotop-1.6.0.tar.gz
cd innotop-1.6.0
perl Makefile.PL
At this point if you get the following output, you are good to continue:
Checking if your kit is complete...
Looks good
Writing Makefile for innotop
If you get something similar to following, you will need to take care of the prerequisites:
Looks good
Warning: prerequisite DBD::mysql 1 not found.
Warning: prerequisite DBI 1.13 not found.
Warning: prerequisite Term::ReadKey 2.1 not found.
Writing Makefile for innotop
Just because they are warnings does not mean you ignore them. So let us install those prerequisites. We will use perl’s cpan shell to get this installed (visit my post on how to install perl modules for more details). If it is your first time starting this up, you will have to answer some questions. Defaults will work fine in all cases.
perl -MCPAN -eshell
install Term::ReadKey
install DBI
install DBD::mysql
Note: you must install DBI before you can install DBD::mysql.
If you get an error like following when you are installing DBD::mysql:
Error: Can't load '/root/.cpan/build/DBD-mysql-4.007/blib/arch/auto/DBD/mysql/mysql.so' for module DBD::mysql: libmysqlclient.so.15: cannot open shared object file: No such file or directory at /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/DynaLoader.pm line 230.
You will have to create a symlink to the object file in your lib64 (or lib if you are not using x64 version) folder:
ln -s /usr/local/mysql/lib/mysql/libmysqlclient.so.15 /lib64/
Once all prerequisites are done, type perl Makefile.PL and you should have no warnings. Continue the install:
make install
At this point you should have innotop installed on your system. Let us do some quick set up so you can start using innotop. We start with configuring your .my.cnf to include connection directives.
vi ~/.my.cnf
Add the following (edit to reflect your install) and save/exit
[mysql]
port = 3306
socket = /tmp/mysql.sock
Start up innotop by typing innotop at your shell prompt. First prompt will ask you to “Enter a name:”. I just put localhost since this will be used to connect locally. Next prompt asks you about DSN entry. I use: DBI:mysql:;mysql_read_default_group=mysql
This tells innotop to read .my.cnf file and use group [mysql] directives. Next prompt is optional (I just press enter). Next two prompts you enter information if you need to.
At this point your innotop installation / testing is complete. You can read man innotop to get more details on how to use innotop.
Thứ Ba, 16 tháng 3, 2010
Multiple PHP Instances With One Apache
Long-winded Introduction
It took me a couple of days to figure this out due to lack of decent tutorials and not enough confidence in my Linux skills to build programs from source. I think I have the hang of it now, and write this up with the intent on providing another, or the only, tutorial on setting up CentOS 5 with multiple instances of PHP using one Apache install. That being said, there are a number of good tutorials out there, just none of them explicitly for CentOS and some leave out some details that n00bs like me get confused about.
PHP4 and PHP5 on SuSE 10.1 – This was by far the most helpful of the tutorials. Even though it was written for SuSE, it works almost straight across for CentOS.
There is also a great list of instructions in the comments on the php.net site under installing PHP for Apache 2.0 on Unix systems (see http://www.php.net/manual/en/install.unix.apache2.php#90478).
I found this one after I wrote up this tutorial at http://cuadradevelopment.com. It’s a bit different, but should work as well.
There are basically two different ways I could have done this. 1- run a single instance of Apache, and run one instance of PHP as a module, and other installs as CGI. 2- run several instances of Apache, each with it’s own instance of PHP as a module. I chose to do the first method for no particular reason. Dreamhost has a post about the good and bad with running PHP as CGI.
So basically, the steps are: 1. Set up Apache and have PHP install as a module. 2. Configure and make another instance of PHP to run as CGI. 3. Add a virtual host to Apache running under a different port to access the PHP as CGI.
Set up Apache with PHP module
So here’s what I did to get the basic Apache, PHP and MySQL working. This sets up the first PHP install to run as a module in Apache:
From a clean install of CentOS 5 (virtually no packages selected during initial install), I installed the following packages:
$ yum install gcc make subversion ImageMagick php php-cli php-common php-ldap php-mysql php-pdo php-pear php-devel bzip2-devel libxml2-devel mysql mysql-server mysql-devel mod_auth_mysql httpd httpd-manual
From there I needed to get PHP 5.2.x, so I did the following to get PHP, Apache, MySQL and PEAR all set up.
1. Step 1: GET PHP 5.2.x
Check out instructions and packages here: http://blog.famillecollet.com/pages/Config-en
$ wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm
$ wget http://rpms.famillecollet.com/el5.i386/remi-release-5-4.el5.remi.noarch.rpm
$ rpm -Uvh remi-release-5.rpm epel-release-5.rpm
$ yum—enablerepo=remi update php-pear php
Copy the /etc/php.ini file from the /etc/php.ini.default:
$ cp /etc/php.ini.default /etc/php.ini
Change the following lines:
* 1. upload_max_filesize = 20M #line 573
* 2. mysql.default_socket =/path/to/mysql/mysql.sock #about line 736
* 3. mysqli.default_socket =/path/to/mysql/mysql.sock #about line 771
2. Step 2: Edit /etc/httpd/conf/httpd.conf by changing the following lines
* 1. Listen xxx.xxx.xxx.xxx:80 #line 134
* 2. ServerAdmin admin@email.org #line 251
* 3. ServerName somesite.org #line 265
* 4. DocumentRoot ”/path/to/htdocs” #line 282
* 5. #line 307
* 6. AllowOverride All #line 328
* 7. DirectoryIndex index.php index.html index.html.var #line 392
3. Step 3: Create the /etc/my.cnf file for MySQL
[mysqld]
datadir=/path/to/mysql
socket=/path/to/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
[client]
socket=/path/to/mysql/mysql.sock
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
4. Step 4: Start apache and mysql and set them to start on boot up:
$ service httpd start
$ service mysqld start
$ chkconfig mysqld on
$ chkconfig httpd on
5. Step 5: Set the MySQL password for root
$ mysqladmin -u root password ‘XXXXXX’
6. Step 6: install Phing and other PEAR packages
$ pear channel-discover pear.phing.info
$ pear channel-discover pear.phpunit.de
$ pear install phing/phing
$ pear install PhpDocumentor
$ pear install XML_Beautifier
$ pear install PHPUnit
$ pecl install Xdebug
Configure second version of PHP
From here we need to install a second version of PHP. Grab the version you want from http://www.php.net/releases/, and stick that any where you want to (usually your root’s home directory is fine). I’m installing PHP 5.2.4, so I’ll use that in my examples.
Unpack the tarball and enter the directory it created.
$ tar -xjf php-5.2.4.tar.bz2
Now, you’ll need to determine which modules you need to install. For this part I used the steps from the php.net comment under “my approach for determining required modules” (copied here, without permission, but with lots of gratitude and full credit going to the author of the comment).
my approach for determining required modules
------------------------------------
1. get the list of the modules
$ php -m | grep -v -e Modules] -e ^$ > php-default-modules
2. create the configure script
$ for i in $(cat php-default-modules); do echo -n "--with-$i ">> phpconfigure.sh ;done
2.2 add #!/bin/bash to the top line, and ./configure to the second line.
Each of the --with-* need to be on the second line.
3. run the configure script, and iterate through the "Configure script errors"
section below until it completes properly
$ ./phpconfigure.sh
4. at the end of the output, look for a notice of unknown options
Notice: Following unknown configure options were used:
--with-date
--with-gum-disease
Check './configure --help' for available options
5. as suggested, execute '$ ./configure --help' and correct the options. The
"for" command above indiscriminately inserts "--with-" for all modules,
but bundled modules may require "--enable-" instead, so mostly you'll
be changing those. For modules that are enabled by default you'll need
to remove the entry.
6. Add anything else you personally want or need. I like to add "--enable-safe-mode".
After doing all of that, I had the following in phpconfigure.sh
#!/bin/bash
./configure –prefix=/usr/share/ –datadir=/usr/share/php –libdir=/usr/share –includedir=/usr/include –bindir=/usr/bin –enable-safe-mode –with-config-file-path=/etc/php542 –enable-force-cgi-redirect –enable-discard-path –with-bz2 –enable-calendar –with-curl –enable-dbase –enable-exif –enable-ftp –with-gettext –with-gmp –with-iconv –with-ldap –with-libxml-dir=/usr/lib/ –enable-mbstring –with-mime_magic –with-mysql –with-mysqli –with-openssl –enable-pcntl –with-pcre-dir=/usr/lib/ –with-pdo_mysql –with-pdo_sqlite –with-readline –enable-shmop –enable-sockets –with-SQLite –enable-wddx –with-xsl –enable-zip –with-zlib
# Changes from what php -m spits out. You don’t need the info below in your phpconfigure.sh script
#–enable-calendar
#–with-ctype # default
#–with-date # not found, default?
#–enable-dbase
#–with-dom # default
#–enable-exif
#–with-filter #default
#–with-ftp
#–with-hash #default
#–with-json #default
#–with-libxml-dir=/usr/lib/
#–enable-mbstring
#–with-memcache #not found, default?
#–enable-pcntl
#–with-pcre-dir=/usr/lib/
#–with-PDO #taken care of with the pdo_mysql and pdo_sqlite
#–with-Reflection #default
#–with-session #default
#–enable-shmop
#–with-SimpleXML #default
#–enable-sockets
#–with-SPL #default
#–with-standard #not found, is it SPL? default?
#–with-tokenizer #default
#–enable-wddx
#–with-xdebug #not found, not needed
#–with-xml #default
#–with-xmlreader #default
#–with-xmlwriter #default
#–enable-zip
#–with-Xdebug #not found, not needed
NOTE: make sure you do not include ‘–with-apxs2=/usr/sbin/apxs’. This is what installs PHP as an Apache module. Also, since you have the original PHP running, you can theoretically make a phpinfo file (with phpinfo() ) in it, and grab the configure entries from that, making sure to change ‘–with-config-file-path=/etc’ ‘–with-config-file-scan-dir=/etc/php.d’
During the configure, you might run into some errors. Again from the php.net comment:
Configure script errors
--------------------------------------------
In my experience, these errors have been due (with any software, PHP included) mostly to missing
development packages, which contain the libraries and headers needed to compile support for that
library's function into the application.
This becomes a process of:
-executing the ./configure script and looking at the error
-installing the devel package providing the resource referenced by the error (google using the error
as search term as needed)
-repeat until the ./configure script makes it through without error
Upshot: identify the software referenced by the error, and install it.
Example
-----------
Example error:
configure: error: Cannot find OpenSSL's
Example explanation
configure is looking for a header (and probably a lot of other stuff) from a missing openssl package.
Example solution:
php-5.2.9]$sudo yum install openssl-devel
The previous yum command should take care of most of those dependencies.
After the phpconfigure script runs without errors, then simply run
$ make
As the JpGraph tutorial explains, there is no need to run “make install”. Just simply copy the php-cgi executable to the proper place. We’ll get to that step shortly.
Set up Apache VirtualHost and website directories
Now you need to create two directories to handle the PHP as CGI. They can be virtually wherever, but should be in the same directory where you have the main html content. So if you set the path to the website data (in the httpd.conf) to /path/to/htdocs/, then you’ll need to make a /path/to/php524/ and a /path/to/php524-cgi/
$ mkdir /path/to/php524/
and
$ mkdir /path/to/php524-cgi/
After you have those directories, you can add the VirtualHost information to the Apache config (httpd.conf). I like to have a separate file for the VirtualHosts, so I added this to the end of the httpd.conf file.
Include conf/XXXXX_vhosts.conf
And to allow VirtualHosts, uncomment this line:
NameVirtualHost *:80
To allow Apache to listen on (or accept requests from) different ports besides the default 80, add another Listen line to the httpd.conf file:
Listen XXX.XXX.XXX.XXX:8524
I used port 8524 to correspond to version 5.2.4 of PHP
Now create the XXXXX_vhosts.conf file
#this doesn’t really seem to be needed, but it’s there
NameVirtualHost *:8524
# this is the original and runs the PHP as a module
DocumentRoot /path/to/htdocs/
ServerName somesite.org
####### Add other Virtual Hosts below here #######
# Setup PHP 5.2.4 on port 8524
DocumentRoot /path/to/php524/
# We use a separate CGI directory
ScriptAlias /cgi-bin/ /etc/php524/
# For good measure we also add recognition of PHP5 index
DirectoryIndex index.php5
# These are the two critical statements for this virtual
# host. This activates PHP 5.2.4 as a CGI module
Action php524-cgi /cgi-bin/php-cgi
AddHandler php524-cgi .php5 .php
#Options None
Options FollowSymLinks
#AllowOverride None
AllowOverride All
Order allow,deny
Allow from all
DirectoryIndex index.html index.php
Now, you need to copy the php-cgi binary/executable to the /path/to/php524-cgi-bin/ directory. The php-cgi file is located in the file where you ran the configure and make for the new php install. So if you did all that in the /opt/php-5.2.4/ directory, the php-cgi will be located at /opt/php-5.2.4/sapi/cgi/php-cgi.
$ cp /opt/php-5.2.4/sapi/cgi/php-cgi /path/to/php524-cgi-bin/
Finally, copy the php.ini file to the right place. And configure as needed.
$ cp /opt/php-5.2.4/php.ini-dist /path/to/php524-cgi-bin/php.ini
Test the apache configs to make sure they work:
$ /usr/sbin/apachectl configtest
If that returns OK restart Apache.
$ /etc/init.d/httpd graceful
You can make a phpinfo page to test that it’s using the new PHP version.
phpinfo();
? >
Then check out your new site: http://somesite.org:8524/phpinfo.php
In order to get the different versions of PHP to interact with MySQL, you’ll have to use the URL on port 80 as the MySQL host. So, for example, in a WordPress install at http://somesite.org:8524/blog, the wp-config.php will have to have the following for the MySQL hostname:
define('DB_HOST', 'dev.omeka.org');
There is some issue with mod_rewrite on the different versions of PHP. I’ll replace this paragraph with a fix when I have one.
UPDATE: 9/9/09 – I figured out how to get the .htaccess working for the Omeka installs we were working with. I needed to change the AllowOverride lines in the vhost.conf (or httpd.conf) file from None, to All.
It took me a couple of days to figure this out due to lack of decent tutorials and not enough confidence in my Linux skills to build programs from source. I think I have the hang of it now, and write this up with the intent on providing another, or the only, tutorial on setting up CentOS 5 with multiple instances of PHP using one Apache install. That being said, there are a number of good tutorials out there, just none of them explicitly for CentOS and some leave out some details that n00bs like me get confused about.
PHP4 and PHP5 on SuSE 10.1 – This was by far the most helpful of the tutorials. Even though it was written for SuSE, it works almost straight across for CentOS.
There is also a great list of instructions in the comments on the php.net site under installing PHP for Apache 2.0 on Unix systems (see http://www.php.net/manual/en/install.unix.apache2.php#90478).
I found this one after I wrote up this tutorial at http://cuadradevelopment.com. It’s a bit different, but should work as well.
There are basically two different ways I could have done this. 1- run a single instance of Apache, and run one instance of PHP as a module, and other installs as CGI. 2- run several instances of Apache, each with it’s own instance of PHP as a module. I chose to do the first method for no particular reason. Dreamhost has a post about the good and bad with running PHP as CGI.
So basically, the steps are: 1. Set up Apache and have PHP install as a module. 2. Configure and make another instance of PHP to run as CGI. 3. Add a virtual host to Apache running under a different port to access the PHP as CGI.
Set up Apache with PHP module
So here’s what I did to get the basic Apache, PHP and MySQL working. This sets up the first PHP install to run as a module in Apache:
From a clean install of CentOS 5 (virtually no packages selected during initial install), I installed the following packages:
$ yum install gcc make subversion ImageMagick php php-cli php-common php-ldap php-mysql php-pdo php-pear php-devel bzip2-devel libxml2-devel mysql mysql-server mysql-devel mod_auth_mysql httpd httpd-manual
From there I needed to get PHP 5.2.x, so I did the following to get PHP, Apache, MySQL and PEAR all set up.
1. Step 1: GET PHP 5.2.x
Check out instructions and packages here: http://blog.famillecollet.com/pages/Config-en
$ wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-2.noarch.rpm
$ wget http://rpms.famillecollet.com/el5.i386/remi-release-5-4.el5.remi.noarch.rpm
$ rpm -Uvh remi-release-5.rpm epel-release-5.rpm
$ yum—enablerepo=remi update php-pear php
Copy the /etc/php.ini file from the /etc/php.ini.default:
$ cp /etc/php.ini.default /etc/php.ini
Change the following lines:
* 1. upload_max_filesize = 20M #line 573
* 2. mysql.default_socket =/path/to/mysql/mysql.sock #about line 736
* 3. mysqli.default_socket =/path/to/mysql/mysql.sock #about line 771
2. Step 2: Edit /etc/httpd/conf/httpd.conf by changing the following lines
* 1. Listen xxx.xxx.xxx.xxx:80 #line 134
* 2. ServerAdmin admin@email.org #line 251
* 3. ServerName somesite.org #line 265
* 4. DocumentRoot ”/path/to/htdocs” #line 282
* 5.
* 6. AllowOverride All #line 328
* 7. DirectoryIndex index.php index.html index.html.var #line 392
3. Step 3: Create the /etc/my.cnf file for MySQL
[mysqld]
datadir=/path/to/mysql
socket=/path/to/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
[client]
socket=/path/to/mysql/mysql.sock
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
4. Step 4: Start apache and mysql and set them to start on boot up:
$ service httpd start
$ service mysqld start
$ chkconfig mysqld on
$ chkconfig httpd on
5. Step 5: Set the MySQL password for root
$ mysqladmin -u root password ‘XXXXXX’
6. Step 6: install Phing and other PEAR packages
$ pear channel-discover pear.phing.info
$ pear channel-discover pear.phpunit.de
$ pear install phing/phing
$ pear install PhpDocumentor
$ pear install XML_Beautifier
$ pear install PHPUnit
$ pecl install Xdebug
Configure second version of PHP
From here we need to install a second version of PHP. Grab the version you want from http://www.php.net/releases/, and stick that any where you want to (usually your root’s home directory is fine). I’m installing PHP 5.2.4, so I’ll use that in my examples.
Unpack the tarball and enter the directory it created.
$ tar -xjf php-5.2.4.tar.bz2
Now, you’ll need to determine which modules you need to install. For this part I used the steps from the php.net comment under “my approach for determining required modules” (copied here, without permission, but with lots of gratitude and full credit going to the author of the comment).
my approach for determining required modules
------------------------------------
1. get the list of the modules
$ php -m | grep -v -e Modules] -e ^$ > php-default-modules
2. create the configure script
$ for i in $(cat php-default-modules); do echo -n "--with-$i ">> phpconfigure.sh ;done
2.2 add #!/bin/bash to the top line, and ./configure to the second line.
Each of the --with-* need to be on the second line.
3. run the configure script, and iterate through the "Configure script errors"
section below until it completes properly
$ ./phpconfigure.sh
4. at the end of the output, look for a notice of unknown options
Notice: Following unknown configure options were used:
--with-date
--with-gum-disease
Check './configure --help' for available options
5. as suggested, execute '$ ./configure --help' and correct the options. The
"for" command above indiscriminately inserts "--with-" for all modules,
but bundled modules may require "--enable-" instead, so mostly you'll
be changing those. For modules that are enabled by default you'll need
to remove the entry.
6. Add anything else you personally want or need. I like to add "--enable-safe-mode".
After doing all of that, I had the following in phpconfigure.sh
#!/bin/bash
./configure –prefix=/usr/share/ –datadir=/usr/share/php –libdir=/usr/share –includedir=/usr/include –bindir=/usr/bin –enable-safe-mode –with-config-file-path=/etc/php542 –enable-force-cgi-redirect –enable-discard-path –with-bz2 –enable-calendar –with-curl –enable-dbase –enable-exif –enable-ftp –with-gettext –with-gmp –with-iconv –with-ldap –with-libxml-dir=/usr/lib/ –enable-mbstring –with-mime_magic –with-mysql –with-mysqli –with-openssl –enable-pcntl –with-pcre-dir=/usr/lib/ –with-pdo_mysql –with-pdo_sqlite –with-readline –enable-shmop –enable-sockets –with-SQLite –enable-wddx –with-xsl –enable-zip –with-zlib
# Changes from what php -m spits out. You don’t need the info below in your phpconfigure.sh script
#–enable-calendar
#–with-ctype # default
#–with-date # not found, default?
#–enable-dbase
#–with-dom # default
#–enable-exif
#–with-filter #default
#–with-ftp
#–with-hash #default
#–with-json #default
#–with-libxml-dir=/usr/lib/
#–enable-mbstring
#–with-memcache #not found, default?
#–enable-pcntl
#–with-pcre-dir=/usr/lib/
#–with-PDO #taken care of with the pdo_mysql and pdo_sqlite
#–with-Reflection #default
#–with-session #default
#–enable-shmop
#–with-SimpleXML #default
#–enable-sockets
#–with-SPL #default
#–with-standard #not found, is it SPL? default?
#–with-tokenizer #default
#–enable-wddx
#–with-xdebug #not found, not needed
#–with-xml #default
#–with-xmlreader #default
#–with-xmlwriter #default
#–enable-zip
#–with-Xdebug #not found, not needed
NOTE: make sure you do not include ‘–with-apxs2=/usr/sbin/apxs’. This is what installs PHP as an Apache module. Also, since you have the original PHP running, you can theoretically make a phpinfo file (with phpinfo() ) in it, and grab the configure entries from that, making sure to change ‘–with-config-file-path=/etc’ ‘–with-config-file-scan-dir=/etc/php.d’
During the configure, you might run into some errors. Again from the php.net comment:
Configure script errors
--------------------------------------------
In my experience, these errors have been due (with any software, PHP included) mostly to missing
development packages, which contain the libraries and headers needed to compile support for that
library's function into the application.
This becomes a process of:
-executing the ./configure script and looking at the error
-installing the devel package providing the resource referenced by the error (google using the error
as search term as needed)
-repeat until the ./configure script makes it through without error
Upshot: identify the software referenced by the error, and install it.
Example
-----------
Example error:
configure: error: Cannot find OpenSSL's
Example explanation
configure is looking for a header (and probably a lot of other stuff) from a missing openssl package.
Example solution:
php-5.2.9]$sudo yum install openssl-devel
The previous yum command should take care of most of those dependencies.
After the phpconfigure script runs without errors, then simply run
$ make
As the JpGraph tutorial explains, there is no need to run “make install”. Just simply copy the php-cgi executable to the proper place. We’ll get to that step shortly.
Set up Apache VirtualHost and website directories
Now you need to create two directories to handle the PHP as CGI. They can be virtually wherever, but should be in the same directory where you have the main html content. So if you set the path to the website data (in the httpd.conf) to /path/to/htdocs/, then you’ll need to make a /path/to/php524/ and a /path/to/php524-cgi/
$ mkdir /path/to/php524/
and
$ mkdir /path/to/php524-cgi/
After you have those directories, you can add the VirtualHost information to the Apache config (httpd.conf). I like to have a separate file for the VirtualHosts, so I added this to the end of the httpd.conf file.
Include conf/XXXXX_vhosts.conf
And to allow VirtualHosts, uncomment this line:
NameVirtualHost *:80
To allow Apache to listen on (or accept requests from) different ports besides the default 80, add another Listen line to the httpd.conf file:
Listen XXX.XXX.XXX.XXX:8524
I used port 8524 to correspond to version 5.2.4 of PHP
Now create the XXXXX_vhosts.conf file
#this doesn’t really seem to be needed, but it’s there
NameVirtualHost *:8524
# this is the original and runs the PHP as a module
DocumentRoot /path/to/htdocs/
ServerName somesite.org
####### Add other Virtual Hosts below here #######
# Setup PHP 5.2.4 on port 8524
DocumentRoot /path/to/php524/
# We use a separate CGI directory
ScriptAlias /cgi-bin/ /etc/php524/
# For good measure we also add recognition of PHP5 index
DirectoryIndex index.php5
# These are the two critical statements for this virtual
# host. This activates PHP 5.2.4 as a CGI module
Action php524-cgi /cgi-bin/php-cgi
AddHandler php524-cgi .php5 .php
#Options None
Options FollowSymLinks
#AllowOverride None
AllowOverride All
Order allow,deny
Allow from all
DirectoryIndex index.html index.php
Now, you need to copy the php-cgi binary/executable to the /path/to/php524-cgi-bin/ directory. The php-cgi file is located in the file where you ran the configure and make for the new php install. So if you did all that in the /opt/php-5.2.4/ directory, the php-cgi will be located at /opt/php-5.2.4/sapi/cgi/php-cgi.
$ cp /opt/php-5.2.4/sapi/cgi/php-cgi /path/to/php524-cgi-bin/
Finally, copy the php.ini file to the right place. And configure as needed.
$ cp /opt/php-5.2.4/php.ini-dist /path/to/php524-cgi-bin/php.ini
Test the apache configs to make sure they work:
$ /usr/sbin/apachectl configtest
If that returns OK restart Apache.
$ /etc/init.d/httpd graceful
You can make a phpinfo page to test that it’s using the new PHP version.
phpinfo();
? >
Then check out your new site: http://somesite.org:8524/phpinfo.php
In order to get the different versions of PHP to interact with MySQL, you’ll have to use the URL on port 80 as the MySQL host. So, for example, in a WordPress install at http://somesite.org:8524/blog, the wp-config.php will have to have the following for the MySQL hostname:
define('DB_HOST', 'dev.omeka.org');
There is some issue with mod_rewrite on the different versions of PHP. I’ll replace this paragraph with a fix when I have one.
UPDATE: 9/9/09 – I figured out how to get the .htaccess working for the Omeka installs we were working with. I needed to change the AllowOverride lines in the vhost.conf (or httpd.conf) file from None, to All.
Thứ Hai, 15 tháng 3, 2010
Logon Type Codes Revealed
Logon Type Codes Revealed
The logon/logoff category of the Windows security log gives you the ability to monitor all attempts to access the local computer. In this article I’ll examine each logon type in greater detail and show you how some other fields in Logon/Logoff events can be helpful for understanding the nature of a given logon attempt.
* Published: Mar 29, 2005
* Updated: Mar 29, 2005
* Section: Articles :: Misc Network Security
* Author: Randall F. Smith
* Printable Version
* Adjust font size: + -
* Rating: 4.6/5 - 500 Votes
* 1
* 2
* 3
* 4
* 5
AddThis
If you want even more advice from Randall F Smith, check out his seminar below:
Attend the only 2-day seminar devoted to the Windows security log
Event IDs 528 and 540 signify a successful logon, event ID 538 a logoff and all the other events in this category identify different reasons for a logon failure. However, just knowing about a successful or failed logon attempt doesn’t fill in the whole picture. Because of all the services Windows offers, there are many different ways you can logon to a computer such as interactively at the computer’s local keyboard and screen, over the network through a drive mapping or through terminal services (aka remote desktop) or through IIS. Thankfully, logon/logoff events specify the Logon Type code which reveals the type of logon that prompted the event.
Logon Type 2 – Interactive
This is what occurs to you first when you think of logons, that is, a logon at the console of a computer. You’ll see type 2 logons when a user attempts to log on at the local keyboard and screen whether with a domain account or a local account from the computer’s local SAM. To tell the difference between an attempt to logon with a local or domain account look for the domain or computer name preceding the user name in the event’s description. Don’t forget that logon’s through an KVM over IP component or a server’s proprietary “lights-out” remote KVM feature are still interactive logons from the standpoint of Windows and will be logged as such.
Logon Type 3 – Network
Windows logs logon type 3 in most cases when you access a computer from elsewhere on the network. One of the most common sources of logon events with logon type 3 is connections to shared folders or printers. But other over-the-network logons are classed as logon type 3 as well such as most logons to IIS. (The exception is basic authentication which is explained in Logon Type 8 below.)
Logon Type 4 – Batch
When Windows executes a scheduled task, the Scheduled Task service first creates a new logon session for the task so that it can run under the authority of the user account specified when the task was created. When this logon attempt occurs, Windows logs it as logon type 4. Other job scheduling systems, depending on their design, may also generate logon events with logon type 4 when starting jobs. Logon type 4 events are usually just innocent scheduled tasks startups but a malicious user could try to subvert security by trying to guess the password of an account through scheduled tasks. Such attempts would generate a logon failure event where logon type is 4. But logon failures associated with scheduled tasks can also result from an administrator entering the wrong password for the account at the time of task creation or from the password of an account being changed without modifying the scheduled task to use the new password.
Logon Type 5 – Service
Similar to Scheduled Tasks, each service is configured to run as a specified user account. When a service starts, Windows first creates a logon session for the specified user account which results in a Logon/Logoff event with logon type 5. Failed logon events with logon type 5 usually indicate the password of an account has been changed without updating the service but there’s always the possibility of malicious users at work too. However this is less likely because creating a new service or editing an existing service by default requires membership in Administrators or Server Operators and such a user, if malicious, will likely already have enough authority to perpetrate his desired goal.
Logon Type 7 – Unlock
Hopefully the workstations on your network automatically start a password protected screen saver when a user leaves their computer so that unattended workstations are protected from malicious use. When a user returns to their workstation and unlocks the console, Windows treats this as a logon and logs the appropriate Logon/Logoff event but in this case the logon type will be 7 – identifying the event as a workstation unlock attempt. Failed logons with logon type 7 indicate either a user entering the wrong password or a malicious user trying to unlock the computer by guessing the password.
Logon Type 8 – NetworkCleartext
This logon type indicates a network logon like logon type 3 but where the password was sent over the network in the clear text. Windows server doesn’t allow connection to shared file or printers with clear text authentication. The only situation I’m aware of are logons from within an ASP script using the ADVAPI or when a user logs on to IIS using IIS’s basic authentication mode. In both cases the logon process in the event’s description will list advapi. Basic authentication is only dangerous if it isn’t wrapped inside an SSL session (i.e. https). As far as logons generated by an ASP, script remember that embedding passwords in source code is a bad practice for maintenance purposes as well as the risk that someone malicious will view the source code and thereby gain the password.
Logon Type 9 – NewCredentials
If you use the RunAs command to start a program under a different user account and specify the /netonly switch, Windows records a logon/logoff event with logon type 9. When you start a program with RunAs using /netonly, the program executes on your local computer as the user you are currently logged on as but for any connections to other computers on the network, Windows connects you to those computers using the account specified on the RunAs command. Without /netonly Windows runs the program on the local computer and on the network as the specified user and records the logon event with logon type 2.
Logon Type 10 – RemoteInteractive
When you access a computer through Terminal Services, Remote Desktop or Remote Assistance windows logs the logon attempt with logon type 10 which makes it easy to distinguish true console logons from a remote desktop session. Note however that prior to XP, Windows 2000 doesn’t use logon type 10 and terminal services logons are reported as logon type 2.
Logon Type 11 – CachedInteractive
Windows supports a feature called Cached Logons which facilitate mobile users. When you are not connected to the your organization’s network and attempt to logon to your laptop with a domain account there’s no domain controller available to the laptop with which to verify your identity. To solve this problem, Windows caches a hash of the credentials of the last 10 interactive domain logons. Later when no domain controller is available, Windows uses these hashes to verify your identity when you attempt to logon with a domain account.
Conclusion
I hope this discussion of logon types and their meanings helps you as you keep watch on your Windows network and try to piece together the different ways users are accessing your computers. Paying attention to logon type is important because different logon types can affect how you interpret logon events from a security perspective. For instance a failed network logon on a server might now be surprising since users must access servers over the network all the time. But a failed network logon attempt in a workstation security log is different. Why is anyone trying to access someone else’s workstation from over the network? As you can see, it pays to understand the security log.
The logon/logoff category of the Windows security log gives you the ability to monitor all attempts to access the local computer. In this article I’ll examine each logon type in greater detail and show you how some other fields in Logon/Logoff events can be helpful for understanding the nature of a given logon attempt.
* Published: Mar 29, 2005
* Updated: Mar 29, 2005
* Section: Articles :: Misc Network Security
* Author: Randall F. Smith
* Printable Version
* Adjust font size: + -
* Rating: 4.6/5 - 500 Votes
* 1
* 2
* 3
* 4
* 5
AddThis
If you want even more advice from Randall F Smith, check out his seminar below:
Attend the only 2-day seminar devoted to the Windows security log
Event IDs 528 and 540 signify a successful logon, event ID 538 a logoff and all the other events in this category identify different reasons for a logon failure. However, just knowing about a successful or failed logon attempt doesn’t fill in the whole picture. Because of all the services Windows offers, there are many different ways you can logon to a computer such as interactively at the computer’s local keyboard and screen, over the network through a drive mapping or through terminal services (aka remote desktop) or through IIS. Thankfully, logon/logoff events specify the Logon Type code which reveals the type of logon that prompted the event.
Logon Type 2 – Interactive
This is what occurs to you first when you think of logons, that is, a logon at the console of a computer. You’ll see type 2 logons when a user attempts to log on at the local keyboard and screen whether with a domain account or a local account from the computer’s local SAM. To tell the difference between an attempt to logon with a local or domain account look for the domain or computer name preceding the user name in the event’s description. Don’t forget that logon’s through an KVM over IP component or a server’s proprietary “lights-out” remote KVM feature are still interactive logons from the standpoint of Windows and will be logged as such.
Logon Type 3 – Network
Windows logs logon type 3 in most cases when you access a computer from elsewhere on the network. One of the most common sources of logon events with logon type 3 is connections to shared folders or printers. But other over-the-network logons are classed as logon type 3 as well such as most logons to IIS. (The exception is basic authentication which is explained in Logon Type 8 below.)
Logon Type 4 – Batch
When Windows executes a scheduled task, the Scheduled Task service first creates a new logon session for the task so that it can run under the authority of the user account specified when the task was created. When this logon attempt occurs, Windows logs it as logon type 4. Other job scheduling systems, depending on their design, may also generate logon events with logon type 4 when starting jobs. Logon type 4 events are usually just innocent scheduled tasks startups but a malicious user could try to subvert security by trying to guess the password of an account through scheduled tasks. Such attempts would generate a logon failure event where logon type is 4. But logon failures associated with scheduled tasks can also result from an administrator entering the wrong password for the account at the time of task creation or from the password of an account being changed without modifying the scheduled task to use the new password.
Logon Type 5 – Service
Similar to Scheduled Tasks, each service is configured to run as a specified user account. When a service starts, Windows first creates a logon session for the specified user account which results in a Logon/Logoff event with logon type 5. Failed logon events with logon type 5 usually indicate the password of an account has been changed without updating the service but there’s always the possibility of malicious users at work too. However this is less likely because creating a new service or editing an existing service by default requires membership in Administrators or Server Operators and such a user, if malicious, will likely already have enough authority to perpetrate his desired goal.
Logon Type 7 – Unlock
Hopefully the workstations on your network automatically start a password protected screen saver when a user leaves their computer so that unattended workstations are protected from malicious use. When a user returns to their workstation and unlocks the console, Windows treats this as a logon and logs the appropriate Logon/Logoff event but in this case the logon type will be 7 – identifying the event as a workstation unlock attempt. Failed logons with logon type 7 indicate either a user entering the wrong password or a malicious user trying to unlock the computer by guessing the password.
Logon Type 8 – NetworkCleartext
This logon type indicates a network logon like logon type 3 but where the password was sent over the network in the clear text. Windows server doesn’t allow connection to shared file or printers with clear text authentication. The only situation I’m aware of are logons from within an ASP script using the ADVAPI or when a user logs on to IIS using IIS’s basic authentication mode. In both cases the logon process in the event’s description will list advapi. Basic authentication is only dangerous if it isn’t wrapped inside an SSL session (i.e. https). As far as logons generated by an ASP, script remember that embedding passwords in source code is a bad practice for maintenance purposes as well as the risk that someone malicious will view the source code and thereby gain the password.
Logon Type 9 – NewCredentials
If you use the RunAs command to start a program under a different user account and specify the /netonly switch, Windows records a logon/logoff event with logon type 9. When you start a program with RunAs using /netonly, the program executes on your local computer as the user you are currently logged on as but for any connections to other computers on the network, Windows connects you to those computers using the account specified on the RunAs command. Without /netonly Windows runs the program on the local computer and on the network as the specified user and records the logon event with logon type 2.
Logon Type 10 – RemoteInteractive
When you access a computer through Terminal Services, Remote Desktop or Remote Assistance windows logs the logon attempt with logon type 10 which makes it easy to distinguish true console logons from a remote desktop session. Note however that prior to XP, Windows 2000 doesn’t use logon type 10 and terminal services logons are reported as logon type 2.
Logon Type 11 – CachedInteractive
Windows supports a feature called Cached Logons which facilitate mobile users. When you are not connected to the your organization’s network and attempt to logon to your laptop with a domain account there’s no domain controller available to the laptop with which to verify your identity. To solve this problem, Windows caches a hash of the credentials of the last 10 interactive domain logons. Later when no domain controller is available, Windows uses these hashes to verify your identity when you attempt to logon with a domain account.
Conclusion
I hope this discussion of logon types and their meanings helps you as you keep watch on your Windows network and try to piece together the different ways users are accessing your computers. Paying attention to logon type is important because different logon types can affect how you interpret logon events from a security perspective. For instance a failed network logon on a server might now be surprising since users must access servers over the network all the time. But a failed network logon attempt in a workstation security log is different. Why is anyone trying to access someone else’s workstation from over the network? As you can see, it pays to understand the security log.
Thứ Hai, 8 tháng 3, 2010
Crontab – Quick & Complete reference Setting up cronjobs in Unix and Linux.
What is crontab?
cron is a unix, solaris utility that allows tasks to be automatically run in the background at regular intervals by the cron daemon.
These tasks are often termed as cron jobs in unix , solaris.
Crontab (CRON TABle) is a file which contains the schedule of cron entries to be run and at specified times.
Crontab Restrictions
You can execute crontab if your name appears in the file /usr/lib/cron/cron.allow. If that file does not exist, you can use
crontab if your name does not appear in the file /usr/lib/cron/cron.deny.
If only cron.deny exists and is empty, all users can use crontab.
If neither file exists, only the root user can use crontab. The allow/deny files consist of one user name per line.
Crontab Location
Where is it stored?
It will be stored /var/spool/cron/ but we only root has permission for those directories
Crontab Commands
export EDITOR=vi ;to specify a editor to open crontab file.
crontab -e Edit your crontab file, or create one if it doesn’t already exist.
crontab -l Display your crontab file.
crontab -r Remove your crontab file.
crontab -v Display the last time you edited your crontab file. (This option is only available on a few systems.)
Crontab file
Crontab syntax :-
A crontab file has five fields for specifying day , date and time followed by the command to be run at that interval.
* * * * * command to be executed
- – – – -
‘ ‘ ‘ ‘ ‘
‘ ‘ ‘ ‘ +—– day of week (0 – 6) (Sunday=0)
‘ ‘ ‘ +——- month (1 – 12)
‘ ‘ +——— day of month (1 – 31)
‘ +———– hour (0 – 23)
+————- min (0 – 59)
* in the value field above means all legal values as in braces for that column.
The value column can have a * or a list of elements separated by commas. An element is either a number in the ranges shown above or two numbers in the range separated by a hyphen (meaning an inclusive range).
Note: The specification of days can be made in two fields: month day and weekday. If both are specified in an entry, they are cumulative meaning both of the entries will get executed .
How to Modify Crontab file?
*
crontab -e
This will open the crontab file and let you edit it. By default this file will be opened with the VI editor and you will need to press the “Insert” key on your keyboard to be able to write in that file.
*
30 13 * * * /home/your_username/run-me.sh >/dev/null 2>&1
The first character you see is “30” this means that crontab will run the script every time the clock hits the 30 minutes mark. Next “13” this means that crontab will run the script when the clock hits 13. The next three * tell crontab to run the script every day, of every month of every weekday. Combining these fields crontab will run the script every day at exactly 13:30. You may notice that we added the “>/dev/null 2>&1” string at the end of the command. The default cron job will always send and e-mail to the root account when ever a command is executed. Now you don’t want to be notified every day that your crontab job has been executed. If you don’t want to receive e-mails every day notifying you about your job’s execution place this “>/dev/null 2>&1” at the end of every instance of every crontab command.
When you are finished adding your commands to the crontab file you need to save and exit. If you are using VI as your editor you need to issue the following commands:
*
Press the Esc (Escape key) on your keyboard to enter the command mode of VI
*
After you pressed Escape then type the following characters :wq! and press Enter. Remember you have to type this characters (remove the quotes): “:wq!”.
linux_crontab_colorful.png
Crontab Example
#A line in crontab file like below removes the tmp files from /home/someuser/tmp each day at 6:30 PM.
30 18 * * * rm /home/someuser/tmp/*
#This runs every fifteen minutes
*/15 * * * * /notesbit/work/scripts/crons/denyhack1 > /dev/null 2>&1
*/20 * * * * /notesbit/work/scripts/crons/denyhack2 > /dev/null 2>&1
#This runs every twenty five minutes
*/25 * * * * /notesbit/work/scripts/crons/denyhack3 > /dev/null 2>&1
*/35 * * * * /notesbit/work/scripts/crons/denyhack4 > /dev/null 2>&1
# This runs every day at 2:14 PM
08 14 * * * /root/work/scripts/crons/stkinc NUAN > /dev/null 2>&1
Crontab Example 1: This crontab example runs updatedb command 35 minutes past every hour.
35 * * * * updatedb
Crontab Example 2: This crontab example runs /usr/local/bin/diskusage.sh every 5 minutes (e.g. 0, 5, 10, 15, …).
*/5 * * * * /usr/local/bin/diskusage.sh
Crontab Example 3: This crontab example runs /usr/local/bin/diskusage.sh at 1:25 AM, 1:50 AM every Tuesday and on 15th of every month.
25,50 1 15 * 2 /usr/local/bin/diskusage.sh
Crontab Example 4: This crontab example runs /usr/local/bin/diskusage.sh at 2:00 PM on 10th of March, June, September and December.
00 14 10 3,6,9,12 * /usr/local/bin/diskusage.sh
Crontab Example 5: This crontab example runs ‘/usr/local/bin/diskusage.sh user@linuxconfig.sh’ at 9.00 PM every Monday, Wednesday, Friday. Note: Using names for the week day and months is extension in some versions of crontab.
00 21 * * Mon,Wed,Fri /usr/local/bin/diskusage.sh user@linuxconfig.sh
Crontab Example 6: This crontab example runs /usr/local/bin/diskusage.sh every 5 minutes during the 5 working days (Monday – Friday), every week and month.
*/5 * * * 1-5 /usr/local/bin/diskusage.sh
Crontab Example 7: This crontab example runs /usr/local/bin/diskusage.sh every minute during the 4-th hour only in Sunday, every week and month. This is every minute from 0:00 till 0:59, then from 4:00 till 4:59, etc.
* */4 * * sun /usr/local/bin/diskusage.sh
3. System wide cron scheduler
As a Linux administrator you can also use predefined cron directories:
/etc/cron.d /etc/cron.daily /etc/cron.hourly /etc/cron.monthly /etc/cron.weekly
If root wishes to run backup.sh script once a week he will place backup.sh script into /etc/cron.weekly directory.
4. Cron Scheduler on user level
Every user can edit, view or remove his own crontab file. If the root user needs to change someone else’s crontab file he must add ‘-u’ option to specify the user name. To edit crontab file for user foobar we can use command:
# crontab -u foobar -e
Remove foobar’s crontab file:
# crontab -u foobar -r
To view foobar’s crontab content:
# crontab -u foobar -l
Crontab Environment
cron invokes the command from the user’s HOME directory with the shell, (/usr/bin/sh).
cron supplies a default environment for every shell, defining:
HOME=user’s-home-directory
LOGNAME=user’s-login-id
PATH=/usr/bin:/usr/sbin:.
SHELL=/usr/bin/sh
Users who desire to have their .profile executed must explicitly do so in the crontab entry or in a script called by the entry
Reference & source: http://www.linuxconfig.org
cron is a unix, solaris utility that allows tasks to be automatically run in the background at regular intervals by the cron daemon.
These tasks are often termed as cron jobs in unix , solaris.
Crontab (CRON TABle) is a file which contains the schedule of cron entries to be run and at specified times.
Crontab Restrictions
You can execute crontab if your name appears in the file /usr/lib/cron/cron.allow. If that file does not exist, you can use
crontab if your name does not appear in the file /usr/lib/cron/cron.deny.
If only cron.deny exists and is empty, all users can use crontab.
If neither file exists, only the root user can use crontab. The allow/deny files consist of one user name per line.
Crontab Location
Where is it stored?
It will be stored /var/spool/cron/ but we only root has permission for those directories
Crontab Commands
export EDITOR=vi ;to specify a editor to open crontab file.
crontab -e Edit your crontab file, or create one if it doesn’t already exist.
crontab -l Display your crontab file.
crontab -r Remove your crontab file.
crontab -v Display the last time you edited your crontab file. (This option is only available on a few systems.)
Crontab file
Crontab syntax :-
A crontab file has five fields for specifying day , date and time followed by the command to be run at that interval.
* * * * * command to be executed
- – – – -
‘ ‘ ‘ ‘ ‘
‘ ‘ ‘ ‘ +—– day of week (0 – 6) (Sunday=0)
‘ ‘ ‘ +——- month (1 – 12)
‘ ‘ +——— day of month (1 – 31)
‘ +———– hour (0 – 23)
+————- min (0 – 59)
* in the value field above means all legal values as in braces for that column.
The value column can have a * or a list of elements separated by commas. An element is either a number in the ranges shown above or two numbers in the range separated by a hyphen (meaning an inclusive range).
Note: The specification of days can be made in two fields: month day and weekday. If both are specified in an entry, they are cumulative meaning both of the entries will get executed .
How to Modify Crontab file?
*
crontab -e
This will open the crontab file and let you edit it. By default this file will be opened with the VI editor and you will need to press the “Insert” key on your keyboard to be able to write in that file.
*
30 13 * * * /home/your_username/run-me.sh >/dev/null 2>&1
The first character you see is “30” this means that crontab will run the script every time the clock hits the 30 minutes mark. Next “13” this means that crontab will run the script when the clock hits 13. The next three * tell crontab to run the script every day, of every month of every weekday. Combining these fields crontab will run the script every day at exactly 13:30. You may notice that we added the “>/dev/null 2>&1” string at the end of the command. The default cron job will always send and e-mail to the root account when ever a command is executed. Now you don’t want to be notified every day that your crontab job has been executed. If you don’t want to receive e-mails every day notifying you about your job’s execution place this “>/dev/null 2>&1” at the end of every instance of every crontab command.
When you are finished adding your commands to the crontab file you need to save and exit. If you are using VI as your editor you need to issue the following commands:
*
Press the Esc (Escape key) on your keyboard to enter the command mode of VI
*
After you pressed Escape then type the following characters :wq! and press Enter. Remember you have to type this characters (remove the quotes): “:wq!”.
linux_crontab_colorful.png
Crontab Example
#A line in crontab file like below removes the tmp files from /home/someuser/tmp each day at 6:30 PM.
30 18 * * * rm /home/someuser/tmp/*
#This runs every fifteen minutes
*/15 * * * * /notesbit/work/scripts/crons/denyhack1 > /dev/null 2>&1
*/20 * * * * /notesbit/work/scripts/crons/denyhack2 > /dev/null 2>&1
#This runs every twenty five minutes
*/25 * * * * /notesbit/work/scripts/crons/denyhack3 > /dev/null 2>&1
*/35 * * * * /notesbit/work/scripts/crons/denyhack4 > /dev/null 2>&1
# This runs every day at 2:14 PM
08 14 * * * /root/work/scripts/crons/stkinc NUAN > /dev/null 2>&1
Crontab Example 1: This crontab example runs updatedb command 35 minutes past every hour.
35 * * * * updatedb
Crontab Example 2: This crontab example runs /usr/local/bin/diskusage.sh every 5 minutes (e.g. 0, 5, 10, 15, …).
*/5 * * * * /usr/local/bin/diskusage.sh
Crontab Example 3: This crontab example runs /usr/local/bin/diskusage.sh at 1:25 AM, 1:50 AM every Tuesday and on 15th of every month.
25,50 1 15 * 2 /usr/local/bin/diskusage.sh
Crontab Example 4: This crontab example runs /usr/local/bin/diskusage.sh at 2:00 PM on 10th of March, June, September and December.
00 14 10 3,6,9,12 * /usr/local/bin/diskusage.sh
Crontab Example 5: This crontab example runs ‘/usr/local/bin/diskusage.sh user@linuxconfig.sh’ at 9.00 PM every Monday, Wednesday, Friday. Note: Using names for the week day and months is extension in some versions of crontab.
00 21 * * Mon,Wed,Fri /usr/local/bin/diskusage.sh user@linuxconfig.sh
Crontab Example 6: This crontab example runs /usr/local/bin/diskusage.sh every 5 minutes during the 5 working days (Monday – Friday), every week and month.
*/5 * * * 1-5 /usr/local/bin/diskusage.sh
Crontab Example 7: This crontab example runs /usr/local/bin/diskusage.sh every minute during the 4-th hour only in Sunday, every week and month. This is every minute from 0:00 till 0:59, then from 4:00 till 4:59, etc.
* */4 * * sun /usr/local/bin/diskusage.sh
3. System wide cron scheduler
As a Linux administrator you can also use predefined cron directories:
/etc/cron.d /etc/cron.daily /etc/cron.hourly /etc/cron.monthly /etc/cron.weekly
If root wishes to run backup.sh script once a week he will place backup.sh script into /etc/cron.weekly directory.
4. Cron Scheduler on user level
Every user can edit, view or remove his own crontab file. If the root user needs to change someone else’s crontab file he must add ‘-u’ option to specify the user name. To edit crontab file for user foobar we can use command:
# crontab -u foobar -e
Remove foobar’s crontab file:
# crontab -u foobar -r
To view foobar’s crontab content:
# crontab -u foobar -l
Crontab Environment
cron invokes the command from the user’s HOME directory with the shell, (/usr/bin/sh).
cron supplies a default environment for every shell, defining:
HOME=user’s-home-directory
LOGNAME=user’s-login-id
PATH=/usr/bin:/usr/sbin:.
SHELL=/usr/bin/sh
Users who desire to have their .profile executed must explicitly do so in the crontab entry or in a script called by the entry
Reference & source: http://www.linuxconfig.org
Thứ Năm, 25 tháng 2, 2010
sudo tips
sudo(8) executes commands as a different user on Unix systems, as allowed by the sudoers configuration file. Commands run via sudo are logged via syslog, providing an audit trail. While sudo may not work on your friends, I consider it essential to system administration.
* sudo homepage.
* Managing /etc/sudoers with CFEngine.
Alternatives
Consider also sudosh, or special logbash versions of the shell that log all commands. Never use the unsafe and unlogged sudo -s, sudo -i, and su commands. Between sudo and proper configuration management, logging in as root should be a very rare occasion.
List Commands
To see what commands can be run on a system, issue sudo -l. Depending on the sudoers configuration, this may prompt for the user’s password.
$ sudo -l
User admin may run the following commands on this host:
(ALL) NOPASSWD: ALL
If root is allowed to run sudo, one can inspect what commands another user may run:
$ sudo sudo -u someotheruser sudo -l
User someotheruser may run the following commands on this host:
(ALL) NOPASSWD: /usr/sbin/cleanup-logs
If administrators are allowed to sudo to any other user, this can be done directly via:
$ sudo -u someotheruser sudo -l
User someotheruser may run the following commands on this host:
(ALL) NOPASSWD: /usr/sbin/cleanup-logs
Configuration
The sudoers configuration file uses Extended Backus-Naur Form (EBNF), which is flexible but complex. For an overview, see the sudoers(5) documentation.
* Always use visudo(8).
The visudo command should be used to edit the sudoers data. Otherwise, errors or permissions problems may crop up randomly. If building a complex sudoers file using configuration management software, sanity check the resulting data with visudo -f tempsudoers -c before moving it into production use.
* Last entry wins
The last matching rule in sudoers wins; that is, if a NOPASSWD entry is followed by an entry that requires the implicit PASSWD, the user will be prompted to enter their password.
ALL ALL=(ALL) NOPASSWD: ALL
ALL ALL=(ALL) ALL
$ sudo -l
User admin may run the following commands on this host:
(ALL) NOPASSWD: ALL
(ALL) ALL
$ sudo -k; sudo /bin/ls
Password:
To avoid this problem, place NOPASSWD entries after any entries that require a password. The following requires passwords for all commands excepting xinetd service changes on a RedHat Linux system:
%wheel ALL=(ALL) ALL
%wheel ALL=NOPASSWD: /sbin/service xinetd *
Disallow Shell Access
Use the following configuration to avoid needless use of unsafe and unlogged shells. Encourage users to avoid launching a root shell, and reserve a special logbash shell that logs all commands for the rare occasions a root shell is needed.
# specify full list of shells and login commands here
Cmnd_Alias SHELLS= /bin/sh, /bin/ksh, /bin/bash, /bin/zsh, \
/bin/csh, /bin/tcsh, \
/usr/bin/login, /usr/bin/su
%wheel ALL=(ALL) ALL, !SHELLS
If the configuration is correct, a user attempting to gain shell access will be properly rejected:
$ sudo -s
Sorry, user jdoe is not allowed to execute '/bin/zsh' as root on …
$ sudo -i
Sorry, user jdoe is not allowed to execute '/bin/sh' as root on …
$ sudo su
Sorry, user jdoe is not allowed to execute '/usr/bin/su' as root on …
* sudo homepage.
* Managing /etc/sudoers with CFEngine.
Alternatives
Consider also sudosh, or special logbash versions of the shell that log all commands. Never use the unsafe and unlogged sudo -s, sudo -i, and su commands. Between sudo and proper configuration management, logging in as root should be a very rare occasion.
List Commands
To see what commands can be run on a system, issue sudo -l. Depending on the sudoers configuration, this may prompt for the user’s password.
$ sudo -l
User admin may run the following commands on this host:
(ALL) NOPASSWD: ALL
If root is allowed to run sudo, one can inspect what commands another user may run:
$ sudo sudo -u someotheruser sudo -l
User someotheruser may run the following commands on this host:
(ALL) NOPASSWD: /usr/sbin/cleanup-logs
If administrators are allowed to sudo to any other user, this can be done directly via:
$ sudo -u someotheruser sudo -l
User someotheruser may run the following commands on this host:
(ALL) NOPASSWD: /usr/sbin/cleanup-logs
Configuration
The sudoers configuration file uses Extended Backus-Naur Form (EBNF), which is flexible but complex. For an overview, see the sudoers(5) documentation.
* Always use visudo(8).
The visudo command should be used to edit the sudoers data. Otherwise, errors or permissions problems may crop up randomly. If building a complex sudoers file using configuration management software, sanity check the resulting data with visudo -f tempsudoers -c before moving it into production use.
* Last entry wins
The last matching rule in sudoers wins; that is, if a NOPASSWD entry is followed by an entry that requires the implicit PASSWD, the user will be prompted to enter their password.
ALL ALL=(ALL) NOPASSWD: ALL
ALL ALL=(ALL) ALL
$ sudo -l
User admin may run the following commands on this host:
(ALL) NOPASSWD: ALL
(ALL) ALL
$ sudo -k; sudo /bin/ls
Password:
To avoid this problem, place NOPASSWD entries after any entries that require a password. The following requires passwords for all commands excepting xinetd service changes on a RedHat Linux system:
%wheel ALL=(ALL) ALL
%wheel ALL=NOPASSWD: /sbin/service xinetd *
Disallow Shell Access
Use the following configuration to avoid needless use of unsafe and unlogged shells. Encourage users to avoid launching a root shell, and reserve a special logbash shell that logs all commands for the rare occasions a root shell is needed.
# specify full list of shells and login commands here
Cmnd_Alias SHELLS= /bin/sh, /bin/ksh, /bin/bash, /bin/zsh, \
/bin/csh, /bin/tcsh, \
/usr/bin/login, /usr/bin/su
%wheel ALL=(ALL) ALL, !SHELLS
If the configuration is correct, a user attempting to gain shell access will be properly rejected:
$ sudo -s
Sorry, user jdoe is not allowed to execute '/bin/zsh' as root on …
$ sudo -i
Sorry, user jdoe is not allowed to execute '/bin/sh' as root on …
$ sudo su
Sorry, user jdoe is not allowed to execute '/usr/bin/su' as root on …
Thứ Hai, 22 tháng 2, 2010
ERROR:- rpmdb: unable to join the environment
PROBLEM:
You get following or error while Rebuilding RPM database:
=======
[root@www root]# rpm –rebuilddb
rpmdb: unable to join the environment
error: db4 error(11) from dbenv->open: Resource temporarily unavailable
error: cannot open Packages index
=======
SOLUTION:
Try This:
=======
[root@www root]# rm -f /var/lib/rpm/__db*
[root@www root]# echo “%__dbi_cdb create private cdb mpool mp_mmapsize=16Mb mp_size=1Mb” > /etc/rpm/macros
[root@www root]# rpm –rebuilddb
=======
You get following or error while Rebuilding RPM database:
=======
[root@www root]# rpm –rebuilddb
rpmdb: unable to join the environment
error: db4 error(11) from dbenv->open: Resource temporarily unavailable
error: cannot open Packages index
=======
SOLUTION:
Try This:
=======
[root@www root]# rm -f /var/lib/rpm/__db*
[root@www root]# echo “%__dbi_cdb create private cdb mpool mp_mmapsize=16Mb mp_size=1Mb” > /etc/rpm/macros
[root@www root]# rpm –rebuilddb
=======
Thứ Năm, 28 tháng 1, 2010
Rename command tips for multiple files
Ddd an extension to multiple files in different location:
$ EXT=`date "+%m%d%y.%H.%M"`
$ find . -type f -name "name" | xargs -i mv {} {}.$EXT
For example rename all *.bak file as *.txt, enter:
$ rename .bak .txt *.bak
Remove all blank space with rename command:
$ rename "s/ *//g" *.mp3
To remove .jpg file extension, you write command as follows:
$ rename 's/\.jpg$//' *.jpg
To convert all uppercase filenames to lowercase:
$ rename 'y/A-Z/a-z/' *
source http://snippets.dzone.com/posts/show/6136
$ EXT=`date "+%m%d%y.%H.%M"`
$ find . -type f -name "name" | xargs -i mv {} {}.$EXT
For example rename all *.bak file as *.txt, enter:
$ rename .bak .txt *.bak
Remove all blank space with rename command:
$ rename "s/ *//g" *.mp3
To remove .jpg file extension, you write command as follows:
$ rename 's/\.jpg$//' *.jpg
To convert all uppercase filenames to lowercase:
$ rename 'y/A-Z/a-z/' *
source http://snippets.dzone.com/posts/show/6136
Thứ Ba, 5 tháng 1, 2010
NET - Files .htaccess và .htpasswd được dùng để làm gì?
Nhiều web servers, trong đó có Apache thường dùng file .htaccess để bảo vệ các thư mục trên web server. Nếu một thư mục nào đó trên web server có lưu file .htaccess, khi bạn duyệt qua thư mục này, web server sẽ popup một hộp thoại yêu cầu nhập username và mật khẩu giống y như lúc bạn gặp Firewall của VDC! Chỉ khi nào bạn nhập vào một username và mật khẩu đúng, bạn mới có thể xem được trong thư mục đó có những gì?!
Danh sách các username và mật khẩu của .htaccess thường được lưu trong file .htpasswd
* Tại sao phải dùng dấu chấm ở trước trong tên file '.htaccess'? Các file có tên bắt đầu là một dấu chấm '.' sẽ được các web servers xem như là các file cấu hình. Các file này sẽ bị ẩn đi(hidden) khi bạn xem qua thư mục đã được bảo vệ bằng file .htaccess
Hướng dẫn sử dụng .htaccess
Bước 1: chuẩn bị username và mật khẩu cho .htaccess
[đt@localhost /]# htpasswd -c ./users vicki
New password: tyt
Re-type new password: tyt
Adding password for user vicki
-c để tạo file mới
Sau khi bạn chạy dòng lệnh trên, trong thư mục hiện tại sẽ xuất hiện file users với nội dung như sau:
vicki:JNSQVx3F3/n0c
File lưu username và mật khẩu có dạng như sau:
:
:
...
:
* Password thường được mã hóa bằng thuật toán DES(Data Encryption Standard). DES được dùng rất phổ biến trên Unix/Linux(*nix), đặc biệt là trong các files /etc/passwd hoặc /etc/shadow. DES rất khó bị crack. Bạn hãy tham khảo một số tài liệu khác để biết rõ về DES.
* Ngoài cách encrypt password bằng htpasswd như trên, bạn cũng có thể dùng Perl code sau để encrypt:
...
$encpass = &encrypt($password);
...
sub encrypt {
my($plain) = @_;
my(@salt);
@salt = ('a'..'z', 'A'..'Z', '0'..'9', '.', '/');
srand(time() ^ ($$ + ($$ << 15)) );
return crypt($plain, $salt[int(rand(@salt))] . $salt[int(rand(@salt))] );
}
Bước 2: tạo 1 file .htaccess với nội dung như sau:
AuthName "Khu vực cấm"
AuthType Basic
AuthUserFile /somepaths/users
require user vicki
# nếu bạn đang dùng Apache Server, hãy thêm các dòng sau vào
# để ngăn chặn users download các files .htaccess & .htpasswd
Order allow,deny
Deny from all
Order allow,deny
Deny from all
Giải thích
AuthName "Khu vực cấm" // tiêu đề của hộp thoại sẽ được popup
AuthUserFile /var/www.users // đường dẫn đầy đủ đến file lưu username & mật khẩu
require user vicki // danh sách các username được phép
Bước 3: tạo 1 thư mục trên web server, chẳng hạn như 'security', upload file .htaccess vào thư mục này, đừng quên chmod 644 cho file .htaccess và users. Thử upload vài files khác vào 'security'
Bước 4: mở trình duyệt web và vào thư mục 'security', http://localhost/security/. Bạn sẽ nhận được một hộp thông báo yêu cầu nhập username & password. Thử dùng username=vicki & password=tyt, bạn sẽ thấy được các files trong 'security' directory.
Hack .htaccess & .htpasswd
1/ Một số người sơ ý không chmod đúng cho files .htaccess và .htpasswd. Vì vậy bạn có thể dễ dàng download chúng về máy tính của mình. Sau đó bạn tìm một công cụ crack DES bằng tự điển như John the Ripper hoặc CrackJack để crack file .htpasswd
2/ Sử dụng các công cụ hack tự động
Bạn có thể sử dụng WWWHack hoặc Brutus để hack các websites được bảo vệ bằng files .htaccess và .htpasswd. Đây là các công cụ tấn công bằng tự điển gọn nhẹ nhưng đa năng, có hổ trợ proxy, rất dễ sử dụng. Tuy nhiên, nếu password quá khó hoặc tốc độ đường truyền của bạn cũng như của websites không tốt, có lẽ bạn phải chờ hàng giờ để WWWHack hoặc Brutus hoàn thành nhiệm vụ. Và bạn cũng đừng quên kiếm một tự điển tốt trước khi tiến hành hack.
* WWWHack và Brutus làm việc như thế nào? DES(Data Encryption Standard) dường như rất khó bị crack, chỉ có một cách duy nhất và cũng là dễ nhất là dùng tự điển. WWWHack và Brutuslàm việc gần giống như nhau. Chúng sẽ chọn một password bất kì trong tự điển, sau đó gởi đến website. Nếu nhận được HTML status code 401 - Authorization Required, "Access Denied", "Enter your password again" có nghĩa là password không đúng, chúng sẽ thử lại với một password khác có trong tự điển. Nếu nhận được HTML status code 200 OK, ... có nghĩa là password đúng, hay nói cách khác là đã bị crack.
(suu tập)
RefLink: http://thegioimang.org/security/files-.htaccess-va-.htpasswd-duoc-dung-de-lam-gi-.html
Danh sách các username và mật khẩu của .htaccess thường được lưu trong file .htpasswd
* Tại sao phải dùng dấu chấm ở trước trong tên file '.htaccess'? Các file có tên bắt đầu là một dấu chấm '.' sẽ được các web servers xem như là các file cấu hình. Các file này sẽ bị ẩn đi(hidden) khi bạn xem qua thư mục đã được bảo vệ bằng file .htaccess
Hướng dẫn sử dụng .htaccess
Bước 1: chuẩn bị username và mật khẩu cho .htaccess
[đt@localhost /]# htpasswd -c ./users vicki
New password: tyt
Re-type new password: tyt
Adding password for user vicki
-c để tạo file mới
Sau khi bạn chạy dòng lệnh trên, trong thư mục hiện tại sẽ xuất hiện file users với nội dung như sau:
vicki:JNSQVx3F3/n0c
File lưu username và mật khẩu có dạng như sau:
:
:
...
:
* Password thường được mã hóa bằng thuật toán DES(Data Encryption Standard). DES được dùng rất phổ biến trên Unix/Linux(*nix), đặc biệt là trong các files /etc/passwd hoặc /etc/shadow. DES rất khó bị crack. Bạn hãy tham khảo một số tài liệu khác để biết rõ về DES.
* Ngoài cách encrypt password bằng htpasswd như trên, bạn cũng có thể dùng Perl code sau để encrypt:
...
$encpass = &encrypt($password);
...
sub encrypt {
my($plain) = @_;
my(@salt);
@salt = ('a'..'z', 'A'..'Z', '0'..'9', '.', '/');
srand(time() ^ ($$ + ($$ << 15)) );
return crypt($plain, $salt[int(rand(@salt))] . $salt[int(rand(@salt))] );
}
Bước 2: tạo 1 file .htaccess với nội dung như sau:
AuthName "Khu vực cấm"
AuthType Basic
AuthUserFile /somepaths/users
require user vicki
# nếu bạn đang dùng Apache Server, hãy thêm các dòng sau vào
# để ngăn chặn users download các files .htaccess & .htpasswd
Order allow,deny
Deny from all
Order allow,deny
Deny from all
Giải thích
AuthName "Khu vực cấm" // tiêu đề của hộp thoại sẽ được popup
AuthUserFile /var/www.users // đường dẫn đầy đủ đến file lưu username & mật khẩu
require user vicki // danh sách các username được phép
Bước 3: tạo 1 thư mục trên web server, chẳng hạn như 'security', upload file .htaccess vào thư mục này, đừng quên chmod 644 cho file .htaccess và users. Thử upload vài files khác vào 'security'
Bước 4: mở trình duyệt web và vào thư mục 'security', http://localhost/security/. Bạn sẽ nhận được một hộp thông báo yêu cầu nhập username & password. Thử dùng username=vicki & password=tyt, bạn sẽ thấy được các files trong 'security' directory.
Hack .htaccess & .htpasswd
1/ Một số người sơ ý không chmod đúng cho files .htaccess và .htpasswd. Vì vậy bạn có thể dễ dàng download chúng về máy tính của mình. Sau đó bạn tìm một công cụ crack DES bằng tự điển như John the Ripper hoặc CrackJack để crack file .htpasswd
2/ Sử dụng các công cụ hack tự động
Bạn có thể sử dụng WWWHack hoặc Brutus để hack các websites được bảo vệ bằng files .htaccess và .htpasswd. Đây là các công cụ tấn công bằng tự điển gọn nhẹ nhưng đa năng, có hổ trợ proxy, rất dễ sử dụng. Tuy nhiên, nếu password quá khó hoặc tốc độ đường truyền của bạn cũng như của websites không tốt, có lẽ bạn phải chờ hàng giờ để WWWHack hoặc Brutus hoàn thành nhiệm vụ. Và bạn cũng đừng quên kiếm một tự điển tốt trước khi tiến hành hack.
* WWWHack và Brutus làm việc như thế nào? DES(Data Encryption Standard) dường như rất khó bị crack, chỉ có một cách duy nhất và cũng là dễ nhất là dùng tự điển. WWWHack và Brutuslàm việc gần giống như nhau. Chúng sẽ chọn một password bất kì trong tự điển, sau đó gởi đến website. Nếu nhận được HTML status code 401 - Authorization Required, "Access Denied", "Enter your password again" có nghĩa là password không đúng, chúng sẽ thử lại với một password khác có trong tự điển. Nếu nhận được HTML status code 200 OK, ... có nghĩa là password đúng, hay nói cách khác là đã bị crack.
(suu tập)
RefLink: http://thegioimang.org/security/files-.htaccess-va-.htpasswd-duoc-dung-de-lam-gi-.html
NET - Subnet Masks
Khi ta chia một Network ra thành nhiều Network nhỏ hơn, các Network nhỏ nầy được gọI là Subnet. Theo quy ước, các địa chỉ IP được chia ra làm ba Class (lớp) như sau:
class A : 255.0.0.0
class B : 255.255.0.0
class C : 255.255.255.0
Subnet Mask của Class A bằng 255.0.0.0 có nghĩa rằng ta dùng 8 bits, tính từ trái qua phải (các bits được set thành 1), của địa chỉ IP để phân biệt các NetworkID của Class A. Trong khi đó, các bits còn sót lại (trong trường hợp Class A là 24 bits đuợc reset thành 0) được dùng để biểu diễn computers, gọi là HostID. Nếu bạn chưa quen cách dùng số nhị phân hãy đọc qua bài Hệ thống số nhị phân.
Subnetting
Hãy xét đến một địa chỉ IP class B, 139.12.0.0, với subnet mask là 255.255.0.0 (có thể viết là: 139.12.0.0/16, ở đây số 16 có nghĩa là 16 bits được dùng cho NetworkID). Một Network với địa chỉ thế nầy có thể chứa 65,534 nodes hay computers (65,534 = (2^16) –2 ) . Đây là một con số quá lớn, trên mạng sẽ có đầy broadcast traffic.
Giả tỉ chúng ta chia cái Network nầy ra làm bốn Subnet. Công việc sẽ bao gồm ba bước:
1) Xác định cái Subnet mask
2) Liệt kê ID của các Subnet mới
3) Cho biết IP address range của các HostID trong mỗi Subnet
Bước 1: Xác định cái Subnet mask
Để đếm cho đến 4 trong hệ thống nhị phân (cho 4 Subnet) ta cần 2 bits. Công thức tổng quát là:
Y = 2^X
mà Y = con số Subnets (= 4)
X = số bits cần thêm (= 2)
Do đó cái Subnet mask sẽ cần 16 (bits trước đây) +2 (bits mới) = 18 bits
Địa chỉ IP mới sẽ là 139.12.0.0/18 (để ý con số 18 thay vì 16 như trước đây). Con số hosts tối đa có trong mỗi Subnet sẽ là: ((2^14) –2) = 16,382. Và tổng số các hosts trong 4 Subnets là: 16382 * 4 = 65,528 hosts.
Bước 2: Liệt kê ID của các Subnet mới
Trong địa chỉ IP mới (139.12.0.0/18) con số 18 nói đến việc ta dùng 18 bits, đếm từ bên trái, của 32 bit IP address để biểu diễn địa chỉ IP của một Subnet.
Subnet mask trong dạng nhị phân Subnet mask
11111111 11111111 11000000 00000000 255.255.192.0
Như thế NetworkID của bốn Subnets mới có là:
Như thế NetworkID của bốn Subnets mới có là:
Subnet Subnet ID trong dạng nhị phân Subnet ID
1 10001011.00001100.00000000.00000000 139.12.0.0/18
2 10001011.00001100.01000000.00000000 139.12.64.0/18
3 10001011.00001100.10000000.00000000 139.12.128.0/18
4 10001011.00001100.11000000.00000000 139.12.192.0/18
Bước 3: Cho biết IP address range của các HostID trong mỗi Subnet
Vì Subnet ID đã dùng hết 18 bits nên số bits còn lại (32-18= 14) được dùng cho HostID.
Nhớ cái luật dùng cho Host ID là tất cả mọi bits không thể đều là 0 hay 1.
Subnet HostID IP address trong dạng nhị phân HostID IP address Range
1 10001011.00001100.00000000.00000001
10001011.00001100.00111111.11111110 139.12.0.1/18 -139.12.63.254/18
2 10001011.00001100.01000000.00000001 10001011.00001100.01111111.11111110 139.12.64.1/18 -139.12.127.254/18
3 10001011.00001100.10000000.00000001
10001011.00001100.10111111.11111110 139.12.128.1/18 -139.12.191.254/18
4 10001011.00001100.11000000.00000001 10001011.00001100.11111111.11111110 139.12.192.0/18 –139.12.255.254
Bạn có để ý thấy trong mỗi Subnet, cái range của HostID từ con số nhỏ nhất (màu xanh) đến con số lớn nhất (màu cam) đều y hệt nhau không?
Bây giờ ta thử đặt cho mình một bài tập với câu hỏi:
Bạn có thể dùng Class B IP address cho một mạng gồm 4000 computers được không? Câu trả lời là ĐƯỢC. Chỉ cần làm một bài toán nhỏ.
Giả tỉ cái IP address là 192.168.1.1. Thay vì bắt đầu với Subnet mask, trước hết chúng ta tính xem mình cần bao nhiêu bits cho 4000 hosts.
Con số hosts ta có thể có trong một network được tính bằng công thức:
Y = (2^X –2)
Nhớ cái luật dùng cho Host ID là tất cả mọi bits không thể đều là 0 hay 1.
4094 = (2^12 –2)
X = 12 , ta cần 12 bits cho HostIDs, do đó Subnet mask sẽ chiếm 20 (=32-12) bits.
Quá trình tính toán nói trên nầy mang tên là Variable Length Subnet Mask(VLSM).
Supernetting
Giả tỉ ta mạng của ta có 3 Subnets:
Accounting: gồm 200 hosts
Finance : gồm 400 hosts
Marketing: gồm 200 hosts
Bạn hòa mạng với Internet và được Internet Service Provider (ISP) cho 4 Class C IP addresses như sau:
192.250.9.0/24
192.250.10.0/24
192.250.11.0/24
192.250.12.0/24
Bạn có 3 segments và bạn muốn mỗi segment chứa một Network.
Bây giờ bạn làm sao?
Địa chỉ IP trong Class C với default subnet mask 24 cho ta con số Hosts tối đa trong mỗi Network là [(2^X) – 2] = (2^8) – 2 = 254. Như thế segments Accounting và Marketing không bị trở ngại nào cả.
Nhưng ta thấy Segment Finance cần thêm 1 bit mới đủ. Ta làm như sau:
Bước 1: Liệt kê Network IP addresses trong dạng nhị phân
192.250.9.0/24 11000000 11111010 00001001 00000000 (1)
192.250.10.0/24 11000000 11111010 00001010 00000000 (2)
192.250.11.0/24 11000000 11111010 00001011 00000000 (3)
192.250.12.0/24 11000000 11111010 00001100 00000000 (4)
Bước 2: Nhận diện network prefix notation
23 bits đầu (từ trái qua phải) của 2 network IP address (2) and (3) đều giống nhau.
Nếu chúng ta thu Subnet mask từ 24 xuống 23 cho (2) và (3) ta sẽ có một Subnet có thể cung cấp 508 hosts.
IP address của mỗi segment trở thành:
Accounting: gồm 200 hosts: 192.250.9.0/24
Finance: gồm 400 hosts: 192.250.10.0/23
Marketing: gồm 200 hosts: 192.250.12.0/24
Bây giờ IP address 192.250.11.0 trở thành một HostID tầm thường trong Subnet 192.250.10.0/23.
Quá trình ta làm vừa qua bằng cách bớt số bits trong Subnet mask khi gom hai hay bốn (v.v..) subnets lại với nhau để tăng con số HostID tối đa trong một Subnet, được gọi là SUPERNETTING.
Supernetting đuợc dùng trong router bổ xung CIDR (Classless Interdomain Routing và VLSM (Variable Length Subnet Mask).
Và luôn luôn nhớ rằng trong internetwork, NETWORK ID phải là địa chỉ độc đáo (unique).
http://thegioimang.org/mang-can-ban/subnet-masks.html
class A : 255.0.0.0
class B : 255.255.0.0
class C : 255.255.255.0
Subnet Mask của Class A bằng 255.0.0.0 có nghĩa rằng ta dùng 8 bits, tính từ trái qua phải (các bits được set thành 1), của địa chỉ IP để phân biệt các NetworkID của Class A. Trong khi đó, các bits còn sót lại (trong trường hợp Class A là 24 bits đuợc reset thành 0) được dùng để biểu diễn computers, gọi là HostID. Nếu bạn chưa quen cách dùng số nhị phân hãy đọc qua bài Hệ thống số nhị phân.
Subnetting
Hãy xét đến một địa chỉ IP class B, 139.12.0.0, với subnet mask là 255.255.0.0 (có thể viết là: 139.12.0.0/16, ở đây số 16 có nghĩa là 16 bits được dùng cho NetworkID). Một Network với địa chỉ thế nầy có thể chứa 65,534 nodes hay computers (65,534 = (2^16) –2 ) . Đây là một con số quá lớn, trên mạng sẽ có đầy broadcast traffic.
Giả tỉ chúng ta chia cái Network nầy ra làm bốn Subnet. Công việc sẽ bao gồm ba bước:
1) Xác định cái Subnet mask
2) Liệt kê ID của các Subnet mới
3) Cho biết IP address range của các HostID trong mỗi Subnet
Bước 1: Xác định cái Subnet mask
Để đếm cho đến 4 trong hệ thống nhị phân (cho 4 Subnet) ta cần 2 bits. Công thức tổng quát là:
Y = 2^X
mà Y = con số Subnets (= 4)
X = số bits cần thêm (= 2)
Do đó cái Subnet mask sẽ cần 16 (bits trước đây) +2 (bits mới) = 18 bits
Địa chỉ IP mới sẽ là 139.12.0.0/18 (để ý con số 18 thay vì 16 như trước đây). Con số hosts tối đa có trong mỗi Subnet sẽ là: ((2^14) –2) = 16,382. Và tổng số các hosts trong 4 Subnets là: 16382 * 4 = 65,528 hosts.
Bước 2: Liệt kê ID của các Subnet mới
Trong địa chỉ IP mới (139.12.0.0/18) con số 18 nói đến việc ta dùng 18 bits, đếm từ bên trái, của 32 bit IP address để biểu diễn địa chỉ IP của một Subnet.
Subnet mask trong dạng nhị phân Subnet mask
11111111 11111111 11000000 00000000 255.255.192.0
Như thế NetworkID của bốn Subnets mới có là:
Như thế NetworkID của bốn Subnets mới có là:
Subnet Subnet ID trong dạng nhị phân Subnet ID
1 10001011.00001100.00000000.00000000 139.12.0.0/18
2 10001011.00001100.01000000.00000000 139.12.64.0/18
3 10001011.00001100.10000000.00000000 139.12.128.0/18
4 10001011.00001100.11000000.00000000 139.12.192.0/18
Bước 3: Cho biết IP address range của các HostID trong mỗi Subnet
Vì Subnet ID đã dùng hết 18 bits nên số bits còn lại (32-18= 14) được dùng cho HostID.
Nhớ cái luật dùng cho Host ID là tất cả mọi bits không thể đều là 0 hay 1.
Subnet HostID IP address trong dạng nhị phân HostID IP address Range
1 10001011.00001100.00000000.00000001
10001011.00001100.00111111.11111110 139.12.0.1/18 -139.12.63.254/18
2 10001011.00001100.01000000.00000001 10001011.00001100.01111111.11111110 139.12.64.1/18 -139.12.127.254/18
3 10001011.00001100.10000000.00000001
10001011.00001100.10111111.11111110 139.12.128.1/18 -139.12.191.254/18
4 10001011.00001100.11000000.00000001 10001011.00001100.11111111.11111110 139.12.192.0/18 –139.12.255.254
Bạn có để ý thấy trong mỗi Subnet, cái range của HostID từ con số nhỏ nhất (màu xanh) đến con số lớn nhất (màu cam) đều y hệt nhau không?
Bây giờ ta thử đặt cho mình một bài tập với câu hỏi:
Bạn có thể dùng Class B IP address cho một mạng gồm 4000 computers được không? Câu trả lời là ĐƯỢC. Chỉ cần làm một bài toán nhỏ.
Giả tỉ cái IP address là 192.168.1.1. Thay vì bắt đầu với Subnet mask, trước hết chúng ta tính xem mình cần bao nhiêu bits cho 4000 hosts.
Con số hosts ta có thể có trong một network được tính bằng công thức:
Y = (2^X –2)
Nhớ cái luật dùng cho Host ID là tất cả mọi bits không thể đều là 0 hay 1.
4094 = (2^12 –2)
X = 12 , ta cần 12 bits cho HostIDs, do đó Subnet mask sẽ chiếm 20 (=32-12) bits.
Quá trình tính toán nói trên nầy mang tên là Variable Length Subnet Mask(VLSM).
Supernetting
Giả tỉ ta mạng của ta có 3 Subnets:
Accounting: gồm 200 hosts
Finance : gồm 400 hosts
Marketing: gồm 200 hosts
Bạn hòa mạng với Internet và được Internet Service Provider (ISP) cho 4 Class C IP addresses như sau:
192.250.9.0/24
192.250.10.0/24
192.250.11.0/24
192.250.12.0/24
Bạn có 3 segments và bạn muốn mỗi segment chứa một Network.
Bây giờ bạn làm sao?
Địa chỉ IP trong Class C với default subnet mask 24 cho ta con số Hosts tối đa trong mỗi Network là [(2^X) – 2] = (2^8) – 2 = 254. Như thế segments Accounting và Marketing không bị trở ngại nào cả.
Nhưng ta thấy Segment Finance cần thêm 1 bit mới đủ. Ta làm như sau:
Bước 1: Liệt kê Network IP addresses trong dạng nhị phân
192.250.9.0/24 11000000 11111010 00001001 00000000 (1)
192.250.10.0/24 11000000 11111010 00001010 00000000 (2)
192.250.11.0/24 11000000 11111010 00001011 00000000 (3)
192.250.12.0/24 11000000 11111010 00001100 00000000 (4)
Bước 2: Nhận diện network prefix notation
23 bits đầu (từ trái qua phải) của 2 network IP address (2) and (3) đều giống nhau.
Nếu chúng ta thu Subnet mask từ 24 xuống 23 cho (2) và (3) ta sẽ có một Subnet có thể cung cấp 508 hosts.
IP address của mỗi segment trở thành:
Accounting: gồm 200 hosts: 192.250.9.0/24
Finance: gồm 400 hosts: 192.250.10.0/23
Marketing: gồm 200 hosts: 192.250.12.0/24
Bây giờ IP address 192.250.11.0 trở thành một HostID tầm thường trong Subnet 192.250.10.0/23.
Quá trình ta làm vừa qua bằng cách bớt số bits trong Subnet mask khi gom hai hay bốn (v.v..) subnets lại với nhau để tăng con số HostID tối đa trong một Subnet, được gọi là SUPERNETTING.
Supernetting đuợc dùng trong router bổ xung CIDR (Classless Interdomain Routing và VLSM (Variable Length Subnet Mask).
Và luôn luôn nhớ rằng trong internetwork, NETWORK ID phải là địa chỉ độc đáo (unique).
http://thegioimang.org/mang-can-ban/subnet-masks.html
NET - Mô hình TCP/IP
Kiến trúc phân tầng của mô hình TCP/IP
Bộ giao thức TCP/IP được phân làm 4 tầng:
Network access Layer: tương ứng với tầng Physical và Datalink của OSI.
Internet Layer: tương ứng với tầng Network của OSI.
Transport Layer: tương ứng với tầng Transport của OSI.
Application Layer: tương ứng với 3 tầng cao nhất(Session, Presentation, Application) trong OSI.
Có nhiều loại giao thức có trong bộ giao thức truyền thống TCP/IP, nhưng có hai giao thức quan trọng nhất được lấy để đặt tên cho bộ giao thức này là TCP(Transmission Control Protocol) và IP(Internet Protocol). Cụ thể sẽ là:
Các giao thức hoạt đông ở tầng Application:
FTP (File transfer Protocol): Giao thức truyền tệp, cho phép người dùng lấy hoặc gửi một tệp tin đến một máy khác.
Telnet: Chương trình mô phỏng thiết bị đầu cuối cho phép người dùng login vào máy chủ từ một máy khác trên mạng.
SMTP (Simple Mail Transfer Protocol): Một giao thức để truyền thư
DNS (Domain Name Service): Dịch vụ tên miền cho phép nhận ra một máy tính từ tên miền của nó thay vì phải đánh vào địa chỉ IP khó nhớ. Nhiều bạn thường nhầm DNS là Domain Name Server – Sai.
SNMP (Simple Network Management Protocol): Giao thức cung cấp các công cụ quản trị mạng.
Các giao thức hoạt đông ở tầng Transport:
UDP (User Datagram Protocol): Giao thức truyền không tin cậy nhưng ưu điểm của nó là nhanh và tiết kiệm.
TCP (Transmission Control Protocol): Cung cấp một phương thức truyền tin cậy
Các giao thức hoạt đông ở tầng Internet:
IP (Internet Protocol): Giao thức Internet, cung cấp các thông tin để làm sao các gói tin có thể đến được đích.
ARP (Address Resolution Protocol): Giao thức chuyển địa chỉ IP thành địa mạng chỉ vật lý
ICMP (Internet Control Message Protocol): Một giao thức thông báo lỗi xảy ra trên đường truyền.
Các công nghệ thường gặp ở tầng vật lý: Ethernet, Token Ring, Token Bus, Fiber.
Cũng giống như mô hình tham chiếu OSI, dữ liệu từ tầng Application đi xuống các tầng dưới, nơi mà mỗi tầng có nhưng định nghĩa riêng về dữ liệu mà nó sử dụng, chúng thêm vào các header của riêng mình trước khi chuyển tiếp xuống tầng tiếp theo, quá trình nhận diễn ra ngược lại.
Qua 2 bài viết của tôi chắc các bạn đã hình dung phần nào về mô hình phân lớp trong mạng, trong các bài viết này tôi mới nói qua về khái niệm còn nếu đi sâu vào từng lớp, từng giao thức thì còn rất nhiều để nói. Chờ ý kiến của các bạn rùi viết tiếp
RefLink: http://thegioimang.org/forum/ccna-ccnp-ccie/1153-mo-hinh-tcp-ip.html
Bộ giao thức TCP/IP được phân làm 4 tầng:
Network access Layer: tương ứng với tầng Physical và Datalink của OSI.
Internet Layer: tương ứng với tầng Network của OSI.
Transport Layer: tương ứng với tầng Transport của OSI.
Application Layer: tương ứng với 3 tầng cao nhất(Session, Presentation, Application) trong OSI.
Có nhiều loại giao thức có trong bộ giao thức truyền thống TCP/IP, nhưng có hai giao thức quan trọng nhất được lấy để đặt tên cho bộ giao thức này là TCP(Transmission Control Protocol) và IP(Internet Protocol). Cụ thể sẽ là:
Các giao thức hoạt đông ở tầng Application:
FTP (File transfer Protocol): Giao thức truyền tệp, cho phép người dùng lấy hoặc gửi một tệp tin đến một máy khác.
Telnet: Chương trình mô phỏng thiết bị đầu cuối cho phép người dùng login vào máy chủ từ một máy khác trên mạng.
SMTP (Simple Mail Transfer Protocol): Một giao thức để truyền thư
DNS (Domain Name Service): Dịch vụ tên miền cho phép nhận ra một máy tính từ tên miền của nó thay vì phải đánh vào địa chỉ IP khó nhớ. Nhiều bạn thường nhầm DNS là Domain Name Server – Sai.
SNMP (Simple Network Management Protocol): Giao thức cung cấp các công cụ quản trị mạng.
Các giao thức hoạt đông ở tầng Transport:
UDP (User Datagram Protocol): Giao thức truyền không tin cậy nhưng ưu điểm của nó là nhanh và tiết kiệm.
TCP (Transmission Control Protocol): Cung cấp một phương thức truyền tin cậy
Các giao thức hoạt đông ở tầng Internet:
IP (Internet Protocol): Giao thức Internet, cung cấp các thông tin để làm sao các gói tin có thể đến được đích.
ARP (Address Resolution Protocol): Giao thức chuyển địa chỉ IP thành địa mạng chỉ vật lý
ICMP (Internet Control Message Protocol): Một giao thức thông báo lỗi xảy ra trên đường truyền.
Các công nghệ thường gặp ở tầng vật lý: Ethernet, Token Ring, Token Bus, Fiber.
Cũng giống như mô hình tham chiếu OSI, dữ liệu từ tầng Application đi xuống các tầng dưới, nơi mà mỗi tầng có nhưng định nghĩa riêng về dữ liệu mà nó sử dụng, chúng thêm vào các header của riêng mình trước khi chuyển tiếp xuống tầng tiếp theo, quá trình nhận diễn ra ngược lại.
Qua 2 bài viết của tôi chắc các bạn đã hình dung phần nào về mô hình phân lớp trong mạng, trong các bài viết này tôi mới nói qua về khái niệm còn nếu đi sâu vào từng lớp, từng giao thức thì còn rất nhiều để nói. Chờ ý kiến của các bạn rùi viết tiếp
RefLink: http://thegioimang.org/forum/ccna-ccnp-ccie/1153-mo-hinh-tcp-ip.html
Thứ Bảy, 2 tháng 1, 2010
LINUX - How to Find Large Files using Linux Command Line
find / -type f -size +20000k -exec ls -lh {} \; | awk ‘{ print $9 “: ” $5 }’
or
find / -type f -size +20000k -exec ls -lh {} ; | awk ‘{ print $9 “: ” $5 }’ > filelist.txt
find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | sort -nrk 2,2
To put the results in a text file.
Found here, a good resource for command line stuff – http://snippets.dzone.com/posts/show/1491
Đăng ký:
Bài đăng (Atom)