Introducing…

The new monitoring system in the Sys Admin Department. Four consecutive monitors with the advance monitoring tools are fully loaded to provide the best monitoring service to the tech team. The monitoring system includes-

1. Smokeping Network Analyzer tool that monitors the VOIP Phones, Workstations, Network Switches and the Internet Sharing Servers and represent the analysis graph.

2. Realtime monitoring graph that reflects the internet connectivity and consumption of all the ISS.

3. Realtime Call Monitor that shows the number of incoming live calls along with the Agent as well as Group Statistics and

4.  Finally the Zabbix Monitor that keeps an eye at the whole galaxy of servers and websites of our company all the time and informs the current status of the clients connected to it.

If so happens, always start asterisk from asterisk user:-


#su asterisk
#asterisk

We should not use any custom user and of course not “root” user. If we start asterisk as any non-asterisk user, various functionalities may be affected like our custom applications may not get permissions to execute asteisk cli commands.

For ex. If we run asterisk as root, then php apps running via apache, when execute asterisk cli commands, get following error:-
Unable to connect to remote asterisk (does /var/run/asterisk/asterisk.ctl exist?)

Run this command
#ls -l /var/run/asterisk/asterisk.ctl
srwxr-xr-x 1 root root 0 Aug 3 04:55 /var/run/asterisk/asterisk.ctl

As shown in the above result, non root users have no write permission.

Hence stop the asterisk using following command:-
#asterisk -rx "stop gracefully"
When asterisk will stop, use above mentioned process of starting asterisk as asterisk user.

Subversion is designed to replace the commonly used CVS (Concurrent Versions System). The biggest change between CVS and Subversion is that Subversion versions the entire repository, instead of individual files.

Subversion offers two types of repository storage — Berkeley DB and FSFS

Berkeley DB – It is a very reliable database system. SVN uses Berkeley DB logging facilities, which means that the database first writes to on-disk log files a description of any modifications it is about to make, and then makes the modification itself. This is to ensure that if anything goes wrong, the database system can back up to a previous checkpoint—a location in the log files known not to be corrupt—and replay transactions until the data is restored to a usable state. But it has some limitations. First, Berkeley DB environments are not portable. We cannot simply copy a Subversion repository that was created on a UNIX system onto a Windows system and expect it to work.

FSFS – In mid 2004 FSFS (Fast Secure File System) repository came into being. FSFS works faster on directories with a large number of files and takes less disk space. An FSFS repository stores a revision tree in a single file. This storage format is portable across different operating systems and isn’t sensitive to CPU architecture. Transactions are created in separate sub directories. When complete, a single transaction file is created and moved to the revisions directory.

Basic SVN commands are:-

svn help
This will make svn list all the available functions.

svn co/checkout <Source repository path> <Destination path>
This command is used to pull an SVN tree from the server. If the directory structure is changed then it may occasionally need to delete the local copy and re-check it out.

svn st/status
Checks the status of the working copy.

‘?‘ Item is not under version control.
‘A‘ Item is scheduled for Addition.
‘D‘ Item is scheduled for Deletion.
‘M‘ Item has been modified.
‘C‘ Item is in conflict with updates received from the repository.
‘I‘ Item is being ignored (e.g. with the svn:ignore property).
‘!‘ Item is missing (e.g. someone has moved or deleted it without using svn). This also indicates that a directory is incomplete (a checkout or update was interrupted).
‘L‘ Item is locked.
‘~‘ Item is versioned as a directory, but has been replaced by a file, or vice versa.

svn add <filename|folder>
Add files, directories to the working copy and schedule them for addition to the repository. They will be added to the repository on next commit.

svn diff <filename>
Shows the difference of all modifications to file.

svn revert <filename>
Reverts back to the previous version and discard all modifications made in file.

svn ci/commit <filename|folder> -m “Comment: Commit message here”
Commit the changes made in file to the repository with new version.

svn up/update <filename>
Bring the current up-to-date working copy to the repository.

svn delete <filename|folder>
Delete files, directories from working copy and schedule them for deletion to the repository. They will be deleted from repository on next commit.

svn cleanup
Cleanup subversion files resulting from escaped processes and crashed. Recursively clean up the working copy.

svn info <filename>
Display information about file or directory. (Date modified, author, revision, path in repository.)

svn mkdir <folder>
Create a new directory under version control.

svn mv/move <folder1> <folder2>
svn mv/move <file-old-name> <file-new-name>
Rename or move a file or directory. Moves/renames file/directory in repository and in local work area.

Following are the steps of configuration of Zabbix Server at Grmtech’s CO office monitoring server:-
1. Download zabbix source files.

cd /usr/src/
wget http://downloads.sourceforge.net/zabbix/zabbix-1.6.2.tar.gz?modtime=1232112875&big_mirror=0
tar -xzf /usr/src/zabbix-1.6.2.tar.gz -C /gt/fsrvthis/

2. Create zabbix database and import the database structure, default templates and other database configurations provided with zabbix source.

mysql -u root -p
create database zabbix;
quit
cd /gt/fsrvthis/zabbix-1.6.2/create/schema/
mysql -u root –force zabbix –password=passwd <mysql.sql
cd /gt/fsrvthis/zabbix-1.6.2/create/data
mysql -u root –force zabbix –password=passwd < data.sql
mysql -u root –force zabbix –password=passwd < images_mysql.sql

3. Compile and build zabbix source files.

cd /gt/fsrvthis/zabbix-1.6.2/
./configure –enable-server –enable-agent –with-mysql –with-net-snmp –with-libcurl
make install

4. Configure zabbix server and agent settings

mkdir /etc/zabbix
cp /gt/fsrvthis/zabbix-1.6.2/misc/conf/zabbix_server.conf
/etc/zabbix/
Set DBPassword=passwd in /etc/zabbix/zabbix_server.conf
cp /gt/fsrvthis/zabbix-1.6.2/misc/conf/zabbix_agentd.conf /etc/zabbix/

5. Create zabbix group and zabbix user. Set zabbix to run under zabbix user and zabbix group.

groupadd zabbix
useradd zabbix -g zabbix -s /sbin/nologin
passwd zabbix
chown zabbix:zabbix /usr/local/sbin/zabbix

6. Install Zabbix as a service

cp /gt/fsrvthis/zabbix-1.6.2/misc/init.d/redhat/8.0/zabbix* /etc/init.d/
chmod 755 /etc/init.d/zabbix*

In /etc/init.d/zabbix_agentd file set progdir=”/usr/local/sbin/”
In /etc/init.d/zabbix_server file set progdir=”/usr/local/sbin/”

7. Auto start the zabbix server and agent when server boots

chkconfig –-add zabbix_agentd
chkconfig –-add zabbix_server
chkconfig –-level 3 zabbix_server on
chkconfig –-level 3 zabbix_agentd on

8. Start and check zabbix server

service zabbix_server start
cat /tmp/zabbix_server.log

9. Do a simple test to see if the server is listening:

telnet localhost 10051

10. Start and check zabbix agentd

service zabbix_agentd start
cat /tmp/zabbix_agentd.log

11. Configure php.ini settings

max_execution_time = 300
memory_limit = 256M
date.timezone = America/Los_Angeles

12. Restart httpd server

service httpd restart

13. Install the web interface zabbix server

cp -r /gt/fsrvthis/zabbix-1.6.2/frontends/php/*    /var/www/html/

14. Open the url http://zabbix_server_ip_address/ in a browser and follow the steps installation
Edit and put the below given code in /var/www/html/conf/zabbix.conf.php

<?php
global $DB;
$DB["TYPE"]= “MYSQL”;
$DB["SERVER"]= “localhost”;
$DB["PORT"]= “0″;
$DB["DATABASE"]= “zabbix”;
$DB["USER"]= “root”;
$DB["PASSWORD"]= “passwd”;
$ZBX_SERVER= “localhost”;
$ZBX_SERVER_PORT= “10051″;
$IMAGE_FORMAT_DEFAULT= IMAGE_FORMAT_PNG;
?>

15. Start zabbix server in distributed mode.

cd /usr/local/sbin
./zabbix_server -n 1

16. Login to zabbix web interface and manage the zabbix server.

Uid: Admin
Pwd: zabbix

Steps of configuring Zabbix agent at any machine to be monitored by Zabbix Server:-

1. Download zabbix source files.

cd /usr/src/
wget http://downloads.sourceforge.net/zabbix/zabbix-1.6.2.tar.gz?modtime=1232112875&big_mirror=0
tar -xzf /usr/src/zabbix-1.6.2.tar.gz -C /gt/fsrvthis/

2. Create zabbix group and zabbix user.

groupadd zabbix
useradd zabbix -g zabbix -s /sbin/nologin
passwd zabbix

3. Compile and build zabbix source files.

cd /gt/fsrvthis/zabbix-1.6.2/
./configure –enable-agent
make install

4. Add the following lines to /etc/services:

zabbix_agent 10050/tcp
zabbix_trap 10051/tcp

5. Configure zabbix agent settings

mkdir /etc/zabbix
cp /gt/fsrvthis/zabbix-1.6.2/misc/conf/zabbix_agentd.conf /etc/zabbix/

6. edit /etc/zabbix/zabbix_agentd.conf and set Zabbix Server ipaddress and Zabbix server hostname. Both should resolve to each other.

7. Start zabbix agentd

cd /usr/local/sbin/
./zabbix_agentd

Helpful links:-
http://www.netadmintools.com/art526.html      #Full setup commands with sample command output
http://www.zabbix.com/wiki/doku.php?id=howto:centosinstall
http://www.zabbix.com/forum/showthread.php?t=8044 #Cut and Paste walkthrough of Zabbix installation on CentOS 5
http://blog.brendon.com/wp-content/uploads/2008/11/install_zabbix.sh #Zabbix Install Script for CentOS 5

Notes:-
For ./configure …. command following may be helpful:-
1)
For “checking for main in -lcurl… no” error
yum remove curl*
yum remove libidn-devel-0.6.5-1.1
yum install curl*
yum install libidn-devel-0.6.5-1.1

2)For “error: Not found Curl library” or “curl-config not found”
yum remove curl*
yum install curl*

3)
For “net-snmp” error :-
yum install net-snmp*

4)

Make sure that php-gd, php-math, php-mysql is installed or else install latest updates of php by the following:-
yum install php*

5)

if error “Can’t open jason object” is seen in /tmp/zabbix_agentd.log file, install php jason library [not yet solved. error is still coming]

Some important locations:
1. /usr/local/sbin contains zabbix_agent, zabbix_agentd, zabbix_get, zabbix_sender and zabbix_server daemon processes.
2. /etc/zabbix contains zabbix_agentd.conf and zabbix_server.conf where parameters for corresponding daemons can be set.
3. /tmp/ contains zabbix_agentd.log and zabbix_server.log files which logs all error entries from corresponding daemon processes.

Errors yet to be solved:-
1. Mysql, httpd, zabbix_server, zabbix_agentd do not auto start on boot
2. /tmp/zabbix_server.log has error: “Send list of active checks to [127.0.0.1] failed: host [ZABBIX Server] not found”
3. /tmp/zabbix_agentd.log has error: Can’t open jason object
4. The application is located at /var/www/html. Change its location to a proper place in filesystem
5.

• 2 Quad core processors: Intel Xeon Quad-Core L5420 2.5GHz 1333MHz 12MB LGA771 CPU
• 32 GB of RAM (getting upgraded to 64 GB of RAM)
• 6 TB of Hard disk
• Gigabyte network
• Chasis: http://www.supermicro.com/products/system/2U/6025/SYS-6025W-NTR+.cfm
• Motherboard: http://www.supermicro.com/products/motherboard/Xeon1333/5400/X7DWN+.cfm

Preparation
LSI storage controller drivers are not available on the Windows XP installation CD, so complete the following preparation before installation:
1. Download the LSI 53C1030 drivers from the LSI Web site.
2. Using MagicISO or other third-party solutions, create a
.flp image containing the LSI Logic drivers.
3. SCP the floppy image to the virtual machine’s ESX host.
If you are using VirtualCenter 2.5, you can use the VIC to upload the file to the desired datastore.
4. Have a Windows XP CD or ISO image that is accessible form the virtual machine.

Pre-installation Virtual Machine Modifications
Make the following modifications to the virtual machine hardware profile before starting the Windows XP
installation:
1 Using the Virtual Infrastructure Client (VIC), connect to VirtualCenter.
2 Locate the virtual machine that was initially created.
3 Edit the following hardware settings:
a Ensure there is a floppy drive present.
b Ensure the floppy drive is configured to connect at Power On.
c If using a floppy image, ensure the Device Type is set to use a floppy image and is pointing to the LSI
Driver image.
d Ensure the CD/DVD drive is present and configured to connect at power on.
e Ensue the CD/DVD Device Type is configured to point at the Windows XP CD or ISO image.

Installation
Once you complete the pre-installation preparation and modifications, you can install Windows XP:
1 From the Virtual Infrastructure client, connect to VirtualCenter.
2 Power on the virtual machine created earlier.
3 Use the console to view the boot process and to send input to the virtual machine.
4 As the Windows Setup process begins, press F6 to add an additional SCSI driver.
This lets you specify the LSI Logic driver on the floppy image.
The Windows setup process copies all the necessary files to the virtual disk. Complete the setup just as you
would for any normal Windows XP installation. Because this image will be used as a template, however, it is
a good idea to make the configuration as generic as possible. For an instance of customization, see “Creating
a Guest Customization” on page 4).
After completing the Windows setup, perform the following tasks before you finalize the image. Some of these
steps will vary from organization to organization, depending on your Windows imaging standards; some are
optional and noted as such. Many can be managed using a group policy (see “Common GPOs for Managing
Virtual Desktops” on page 5).

Following are steps used to setup a Master-Slave replication :-
1. Assumptions:

1. CentOS 5.2
2. Mysql 5.0
3. • DB1: DbGrmtech
   • Loc1: 10.1.31.9
   • DB2: DbGrmtech
   • Loc2: 10.1.31.6
4. Loc1 is master and Loc2 is slave

2. Create a dump file of DB1 using the command:
mysqldump –port 3306 -u GrmtechUser –password=GrmtechPass -h 10.1.31.9 –databases DbGrmtech –add-drop-table > dbdump.sql

3. Rsync the dbdump file to Loc2 using the command:
rsync -azpSv -e ’ssh -p 99′ dbdump.sql 10.1.31.6:/home/user/domains/dbbackup/grmtech/dbdump.sql

4. Import the dbdump file in Loc2 using the command:
mysql -u GrmtechUser –force DbGrmtech –password=GrmtechPass -h 10.1.31.6 < dbdump.sql

5. Edit Loc1’s /etc/my.cnf file and add the following lines under [mysqld]:
server-id = 1
log-bin = /var/log/mysql/bin.log
binlog-do-db=DbGrmtech

6. Give the command to restart mysql daemon:
service mysqld restart

7. Login to mysql at Loc2 and give the following command:
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO ‘crmuser’@'10.1.31.6′ IDENTIFIED BY ‘crmpass’

8. Edit Loc2’s /etc/my.cnf file and add the following lines under [mysqld]:
server-id = 2
master-host = 10.1.31.9
master-port = 3306
master-user = crmuser
master-password = crmpass

replicate-wild-do-table=DbGrmtech.%
replicate-ignore-table = DbGrmtech.tracker

log-bin = /var/log/mysql/bin.log
log-bin-index = /var/log/mysql/log-bin.index
log-error = /var/log/mysql/error.log

relay-log = /var/log/mysql/relay.log
relay-log-info-file = /var/log/mysql/relay-log.info
relay-log-index = /var/log/mysql/relay-log.index

9. Give the command to restart mysql daemon:
service mysqld restart

10. Login to mysql at Loc2 and give the following commands:
Show Slave Status;
If master host setting is not what u set at my.cnf, use this command at mysql prompt:
CHANGE MASTER TO MASTER_HOST=’10.1.31.9′, MASTER_USER=’crmuser’, MASTER_PASSWORD=’crmpass’, MASTER_LOG_FILE=’bin.000001′;

By doing the above settings the changes at DB1@Loc1 will be auto replicated to DB2@Loc2 but not vice versa.

Some commands handy in troubleshooting the entire setup:
At Mysql prompt
1. SHOW MASTER STATUS
2. SHOW SLAVE STATUS
3. START/STOP MASTER
4. START/STOP SLAVE

At command prompt
1. tail -50 /var/log/mysql/bin.000001: to see latest changes in DB1

Helpful links:
1. http://www.onlamp.com/pub/a/onlamp/2005/06/16/MySQLian.html

Network Load Balancing, a clustering technology included in the Microsoft Windows 2000 Advanced Server and Datacenter Server operating systems, enhances the scalability and availability of mission-critical, TCP/IP-based services, such as Web, Terminal Services, virtual private networking, and streaming media servers. This component runs within cluster hosts as part of the Windows 2000 operating system and requires no dedicated hardware support. To scale performance, Network Load Balancing distributes IP traffic across multiple cluster hosts. It also ensures high availability by detecting host failures and automatically redistributing traffic to the surviving hosts. Network Load Balancing provides remote controllability and supports rolling upgrades from the Windows NT 4.0 operating system.

The unique and fully distributed architecture of Network Load Balancing enables it to deliver very high performance and failover protection, especially in comparison with dispatcher-based load balancers.

Advantages

Network Load Balancing is superior to other software solutions such as round robin DNS (RRDNS), which distributes workload among multiple servers but does not provide a mechanism for server availability. If a server within the host fails, RRDNS, unlike Network Load Balancing, will continue to send it work until a network administrator detects the failure and removes the server from the DNS address list. This results in service disruption for clients. Network Load Balancing also has advantages over other load balancing solutions—both hardware- and software-based—that introduce single points of failure or performance bottlenecks by using a centralized dispatcher. Because Network Load Balancing has no proprietary hardware requirements, any industry-standard compatible computer can be used. This provides significant cost savings when compared to proprietary hardware load balancing solutions.

The unique and fully distributed software architecture of Network Load Balancing enables it to deliver the industry’s best load balancing performance and availability. The specific advantages of this architecture are described below in the “Network Load Balancing Architecture” section.

Network Load Balancing Architecture

To maximize throughput and high availability, Network Load Balancing uses a fully distributed software architecture. An identical copy of the Network Load Balancing driver runs in parallel on each cluster host. The drivers arrange for all cluster hosts on a single subnet to concurrently detect incoming network traffic for the cluster’s primary IP address (and for additional IP addresses on multihomed hosts). On each cluster host, the driver acts as a filter between the network adapter’s driver and the TCP/IP stack, allowing a portion of the incoming network traffic to be received by the host. By this means incoming client requests are partitioned and load-balanced among the cluster hosts.

Network Load Balancing runs as a network driver logically situated beneath higher-level application protocols, such as HTTP and FTP. Figure 2 below shows the implementation of Network Load Balancing as an intermediate driver in the Windows 2000 network stack.

Figure: Network Load Balancing runs as an intermediate driver between the TCP/IP protocol and network adapter drivers within the Windows 2000 protocol stack Note that although two network adapters are shown, only one adapter is needed to use Network Load Balancing.
This architecture maximizes throughput by using the broadcast subnet to deliver incoming network traffic to all cluster hosts and by eliminating the need to route incoming packets to individual cluster hosts. Since filtering unwanted packets is faster than routing packets (which involves receiving, examining, rewriting, and resending), Network Load Balancing delivers higher network throughput than dispatcher-based solutions. As network and server speeds grow, its throughput also grows proportionally, thus eliminating any dependency on a particular hardware routing implementation. For example, Network Load Balancing has demonstrated 250 megabits per second (Mbps) throughput on Gigabit networks.

Network Load Balancing Performance

The performance impact of Network Load Balancing can be measured in four key areas:

• CPU overhead on the cluster hosts, which is the CPU percentage required to analyze and filter network packets (lower is better).
• Response time to clients, which increases with the non-overlapped portion of CPU overhead, called latency (lower is better).
• Throughput to clients, which increases with additional client traffic that the cluster can handle prior to saturating the cluster hosts (higher is better).
• Switch occupancy, which increases with additional client traffic (lower is better) and must not adversely affect port bandwidth.

In addition, Network Load Balancing’s scalability determines how its performance improves as hosts are added to the cluster. Scalable performance requires that CPU overhead and latency not grow faster than the number of hosts.

Options are available for Linux based Networks too. We just need to explore them.

This was just a small overview. For the complete technical details, please click on the link below-

http://technet.microsoft.com/en-us/library/bb742455.aspx

The virtual appliance is composed of:
* CentOS 5.2
* VMware Tools
* Squid 3
* DansGuardian

Click on below links to download the virtual appliance.
1. centos-dansguardian-virtualappliance.vmx
2. centos-dansguardian-virtualappliance.vmdk

You can use this virtual appliance to do things like:
1. Do phrase matching, PICS filtering and URL filtering.
2. Restrict list of sites.

Username: root
Password: dans12

Steps to configure the virtual appliance:-
1. yum install squid*

2. /etc/rc.d/init.d/squid restart

3. Enable packet forwarding in /etc/sysctl.conf. Set net.ipv4.ip_forward = 1.

4. Make changes to /etc/squid/squid.conf. A copy of the original files are kept with the extension .orig *

5. Yum install dansguardian or do the ./configure if you want the 2.10.0 version

6. Make changes in /etc/dansguardian/dansguardian.conf. A copy of the original files are kept with the extension .orig *

7. Enable A. squid B. iptables C. dansguardian using ntsysv

8. Configure iptables to redirect requests from port 80 to port 8080 using following commands**:-
a. iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp -d ! <ip-address-of-i-interface> –dport 80 -j REDIRECT –to-ports 8080
b. iptables-save > /etc/sysconfig/iptables
c. service iptables restart

Improving the performance of squid:
Edit squid.conf with these settings:-
maximum_object_size_in_memory 8 MB earlier it was 8KB
cache_mem 800 MB earlier it was 8 MB

Improving the performance of dansguardian:
Edit dansguardian.conf with these settings:-
urlcachenumber = 5000  (default is 1000)
urlcacheage = 9000 (default is 900)

* The difference between original and copied file can be seen using “diff orig_file new_file”
** This will make sure that requests for web sites hosted on this server will not get redirected to port 8080

Desktop publishing is of two types:
1. Thin client – The desktop is running from the server and only the display is happening on the client.
2. Thick Client – The desktop and its display is running on the client.

Software we use at Grmtech for desktop publishing is Vmware.

Application publishing is when people log into the server to use the same app every time.
Software we use at Grmtech for Application publishing is Windows 2003 server.