Jan 26, 2013

Open Source Communities


#1 First Berkeley Software Distribution (1BSD) Unix


Came into existence: March 9, 1978


It was not Linux which that was the first open-source operating system; the honor goes to Bill Joy’s first version of BDS Unix. When Unix first showed up in 1969, it was open source, it was closed later, but it kept the free software flag flying. Though Linux enjoyed the lime light, Unix was important too.


BSD is a Unix operating system derivative developed and distributed by the Computer Systems Research Group (CSRG) of the University of California, Berkeley. Today the term "BSD" is often used non-specifically to refer to any of the BSD descendants which together form a branch of the family of Unix-like operating systems. Operating systems derived from the original BSD code remain actively developed and widely used.




#2 VistA


Came into existence: 1975


The oldest open-source program which is still commonly used is VistA. The Veterans Health Information Systems and Technology Architecture (VistA) is an enterprise-wide information system built around an Electronic Health Record (EHR), used throughout the United States Department of Veterans Affairs (VA) medical system, known as the Veterans Health Administration (VHA).


#3 BRL-CAD


Came into existence: Dec 16, 1983


BRL-CAD, Ballistic Research Laboratory-computer-aided design, is a constructive solid geometry (CSG) solid modeling computer-aided design (CAD) system. It includes an interactive geometry editor, ray tracing support for graphics rendering and geometric analysis, computer network distributed framebuffer support, scripting, image-processing and signal-processing tools. This program, which is still being worked on today, is used by the U.S. military to model ballistic attacks on vehicles.



#4 X Window System


Came into existence: 1983


At the same time that Emacs and gcc started to roll, others at MIT were working on the X Window System. The X Window System is a computer software system and network protocol that provides a basis for graphical user interfaces (GUIs) and rich input device capability for networked computers.  No one knew it at the time that X Window would eventually become the basis for all important Linux and Unix interfaces, and the foundation for the Mac OS X 



#5 GNU Emacs


Came into existence: 1984


Emacs and its derivatives are a family of text editors that are characterized by their extensibility. GNU Emacs is an extensible, customizable text editor—and more. At its core is an interpreter for Emacs Lisp, a dialect of the Lisp programming language with extensions to support text editing.



#6 GNU C compiler (gcc)


Came into existence: March 22, 1987


The GNU Compiler Collection (GCC) is a compiler system produced by the GNU Project supporting various programming languages. GCC has been adopted as the standard compiler by most other modern Unix-like computer operating systems, including Linux, and the BSD family. GCC is also available for most embedded platforms, including Symbian (called gcce), AMCC and Freescale Power Architecture-based chips. The compiler can target a wide variety of platforms, including videogame consoles such as the PlayStation 2 and Dreamcast.


Stallman was listed as the author but cited others for their contributions, including Jack Davidson and Christopher Fraser for the idea of using RTL as an intermediate language, Paul Rubin for writing most of the preprocessor and Leonard Tower for "parts of the parser, RTL generator, RTL definitions, and of the Vax machine description."




#7 Perl


Came into existence: December 18, 1987


Perl is a high-level, general-purpose, interpreted, dynamic programming language. Though Perl is not officially an acronym, it is considered to stand for “Practical Extraction and Reporting Language.” Perl was originally developed by Larry Wall in 1987 as a general-purpose Unix scripting language to make report processing easier. Since then, it has undergone many changes and revisions.


Perl borrows features from other programming languages including C, shell scripting (sh), AWK, and sed. The language provides powerful text processing facilities without the arbitrary data-length limits of many contemporary Unix tools, facilitating easy manipulation of text files.


Perl gained widespread popularity in the late 1990s as a CGI scripting language, in part due to its parsing abilities. In addition to CGI, Perl is used for graphics programming, system administration, network programming, finance, bioinformatics, and other applications.


#8 GNU C Library (glibc)


Came into existence: February 1988


C programming language remains a vital language in open-source circles and to do much with C you need a good, general purpose library. The GNU C Library, commonly known as glibc, is the GNU Project's (The GNU Project is a free software, mass collaboration project, announced on 27 September 1983, by Richard Stallman at MIT) implementation of the C standard library. glibc was written by Roland McGrath, working for the Free Software Foundation (FSF) in the 1980s. By early 1988, McGrath had given what would become glibc, "a nearly complete set of ANSI C library functions." From this work would spring innumerable programs including Linux.



#9 Python


Came in to existence: February 20, 1991


Guido van Rossum, Python's creator, began work on this importantprogramming language in December 1989. It wasn't until February 1991, that it saw the light. Python is a general-purpose, interpreted high-level programming language whose design philosophy emphasizes code readability. Its syntax is said to be clear and expressive. Python has a large and comprehensive standard library.


Python supports multiple programming paradigms, including object-oriented, imperative and functional programming styles. It features a fully dynamic type system and automatic memory management, similar to that of Scheme, Ruby, Perl, and Tcl. Like other dynamic languages, Python is often used as a scripting language, but is also used in a wide range of non-scripting contexts.


According to Black Duck Software's Ohloh programming statistics, "Python is the fastest growing language in the open source world as measured by number of contributors."


Even Microsoft provides a Python IDE (integrated development environment ) for Visual Studio.





#10 Linux


Came into existence: August 25, 1991


Linux is a Unix-like computer operating system. The defining component of Linux is the Linux kernel, an operating system kernel first released in 5 October 1991 by Linus Torvalds. Torvalds wrote "I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones" to the Minix Usenet group, and unknowingly, started the operating system juggernaut. This OS later became the most successful of open-source programs.


It is a leading operating system on servers and other big iron systems such as mainframe computers and supercomputers. Linux also runs on embedded systems such as mobile phones, tablet computers, network routers, televisions and video game consoles; the Android system in wide use on mobile devices is built on the Linux kernel.

Oct 12, 2012

Configure Software Raid in Linux


The term RAID is an acronym for the phrase, Redundant Array of Independent Disks. RAID is a way of combining the storage available across multiple disks and supplying users a single, unified virtual device.
RAID can be used to provide:
  • data integrity
  • fault tolerance
  • improved performance
  • greater storage capacity
Configuring Software RAID
Configuring RAID using Linux requires a number of steps that need to be followed carefully. In the tutorial example, we will be configuring RAID 5 using a system with three pre-partitioned hard disks. The partitions to be used are:

/dev/hde1
/dev/hdf2
/dev/hdg1

RAID Partitioning
You first need to identify two or more partitions, each on a separate disk. If you are doing RAID 0 or RAID 5, the partitions should be of approximately the same size, as in this scenario. RAID limits the extent of data access on each partition to an area no larger than that of the smallest partition in the RAID set.

Determining Available Partitions
First use the fdisk -l command to view all the mounted and unmounted filesystems available on your system. You may then also want to use the df -k command, which shows only mounted filesystems but has the big advantage of giving you the mount points too.

These two commands should help you to easily identify the partitions you want to use. Here is some sample output of these commands.
[root@linuxbox tmp]# fdisk -l
Disk /dev/hda: 12.0 GB, 12072517632 bytes
255 heads, 63 sectors/track, 1467 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/hda: 12.0 GB, 12072517632 bytes
255 heads, 63 sectors/track, 1467 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device BootStartEndBlocksIdSystem
/dev/hda111310439183Linux
/dev/hda2141441052257+83Linux
/dev/hda3145209522112+82Linux swap
/dev/hda42101467101048855Extended
/dev/hda52106553582463+83Linux
...     
...     
/dev/hda1514551467104391  

[root@linuxbox tmp# df -k
Filesystem1K-blocksUsedAvailableUse%Mounted on
/dev/hda21035692 163916 81916417%/
/dev/hda1 1010868357875109%/boot
/dev/hda15  1010864127917405%/data1
...     
...     
...     
/dev/hda75336664464228460134410%/var
Unmount the Partitions
You don't want anyone else accessing these partitions while you are creating the RAID set, so you need to make sure they are unmounted.

[root@linuxbox tmp]# umount /dev/hde1
[root@linuxbox tmp]# umount /dev/hdf2
[root@linuxbox tmp]# umount /dev/hdg1

Prepare The Partitions With FDISK
You have to change each partition in the RAID set to be of type FD (Linux raid autodetect), and you can do this with fdisk. Here is an example using /dev/hde1.

[root@linuxbox tmp]# fdisk /dev/hde
The number of cylinders for this disk is set to 8355.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help):
Use FDISK Help
Now use the fdisk m command to get some help:
Command (m for help): m
...
...
p   print the partition table
q   quit without saving changes
s   create a new empty Sun disklabel
t   change a partition's system id
...
...
Command (m for help):





RAID in Linux & RAID Levels


RAID in Linux

The term RAID is an acronym for the phrase, Redundant Array of Independent Disks. RAID is a way of combining the storage available across multiple disks and supplying users a single, unified virtual device.
RAID can be used to provide:
  • data integrity
  • fault tolerance
  • improved performance
  • greater storage capacity
Hard disks are mechanical devices involving moving parts and unfortunately tend to fail over time. There are also physical limits to the speed at which data can be read and/or written to disks. RAID helps mitigate this risk by protecting data stored on hard disks and improving disk performance by writing the data to multiple

Physical locations according to several different schemas, known as "RAID Levels". Furthermore, RAID can be provided by either dedicated, specialized hardware or by the operating system at a virtual layer.

Hardware RAID solutions exist that operate as dedicated devices, usually as PCI expansion cards or directly on the motherboard. The independent disks attach to the hardware interface. In a true hardware RAID, the operating system simply writes data to the hardware RAID controller which handles the multiplicitous reads and writes to the associated disks. Other so−called hardware RAIDs rely on special drivers to the operating system; these act more like software RAIDs in practice. With current technology, hardware RAIDconfigurations are generally chosen for very large RAIDs.

Additionally, some operating systems, including Linux®, provide RAID functionality within a software layer.RAID partitions are logically combined and a virtual device appears to higher layers of the operating system in place of the multiple constituent devices. This solution is often a high−performance and inexpensive alternative available for RAID users.

RAID levels
There are many RAID levels. It will be impossible to list them all here. In this classroom, we will mention the most common, most important RAID types, all of which are fully supported by Linux.

RAID 0 (Striping)
This level is achieved by grouping 2 or more hard disks into a single unit with the total size equaling that of all disks used.

Practical example: 3 disks, each 80GB in size can be used in a 240GB RAID 0 configuration.

RAID 0 works by breaking data into fragments and writing to all disk simultaneously. This significantly improves the read and write performance.

On the other hand, no single disk contains the entire information for any bit of data committed. This means that if one of the disks fails, the entire RAID is rendered inoperable, with unrecoverable loss of data.

RAID 0 is suitable for non-critical operations that require good performance, like the system partition or the /tmp partition where lots of temporary data is constantly written. It is not suitable for data storage. 
Uasable Space in Raid level0 = (smallest disk) * (no. of disks)

Kernel in Linux, kernel services & configurations

Kernel :- kernel is the heart of the operating system. It manages communication with hardware, decides which processes to run, and provides each process with an isolated, virtual address space in which to run. The kernel is what the GRUB boot loader loads into memory. The kernel loads device driver modules. It also allocates hardware resources such as IRQ ports, I/O addresses, and DMA channels. A recompiled kernel can lead to:


  • Greatly improved speed at which kernel services operate.
  • Direct support for commonly used drivers.
  • Dynamic loading of appropriate drivers as modules.
  • Lower memory consumption by removing unneeded components.
  • Support for high-end hardware, such as memory above 4GB,hardware array controllers, symmetric multiprocessing (multiple CPU) support, and more.

Kernels can be organized as one big unit or as a lot of interconnected pieces. Kernels are called up by boot loaders when you start your system.

Monolithic Versus Modular
A monolithic kernel is a kernel in which all the device modules are built directly into the kernel.

Modular kernels have many of their devices built as separate loadable modules. Monolithic kernels can communicate with devices faster, since the kernels can talk to the hardware only indirectly through a module table. Unfortunately, the typical monolithic kernel is huge, which reduces available RAM. In addition, some systems just can't boot a kernel that's too large.

Linux once had problems loading modular kernels for some hardware. With a monolithic kernel, the drivers are already there and are often more appropriate for certain components such as embedded hardware.

A modular kernel has greater flexibility. We can compile almost all drivers as modules, and then each module can be inserted into the kernel whenever you need it. Modules keep the initial kernel size low, which decreases the boot time and improves overall performance. If Linux has trouble loading a kernel module, We can use the modprobe or insmod command to load modules as needed, and add those options to the /etc/modprobe.conf file.(in Centos5/Rhel5).

Upgrading  Kernel :-
Updating the kernel is not as difficult as it looks. We should never overwrite or upgrade an existing kernel, as mistakes happen. New kernels are handled by installing the newly built kernel in /boot and then adding another boot option to your boot loader configuration file (/boot/grub/grub.conf) for the new kernel. GRUB treats the new kernel as if it were an entirely new operating system. If we install the new kernel directly from a Red Hat configured RPM, it updates the boot loader automatically.

If you do make a drastic mistake and the kernel doesn't boot, you can simply reboot the system and select the old kernel from the GRUB menu.

There are different versions of kernel-devel, kernel-PAE, kernel-xen, and kernel-headers packages for each supported architecture.

Available  Linux Kernels (and Related Packages)
Kernel RPM
Description / Architecture
kernel-versionnum.i686
Designed for PCs with a single Intel/AMD CPU; also works with dual-core systems
kernel-versionnum.ia64
Designed for Itanium2 systems
kernel-devel-versionnum
Installs drivers and other information to help compile third-party drivers
kernel-PAE-versionnum
If you have more than 4GB of RAM, install the PAE kernel associated with your CPU architecture
kernel-PAE-devel-versionnum
If you have more than 4GB of RAM, install the PAE kernel associated with your CPU architecture
kernel-headers-versionnum
Includes kernel headers; often sufficient for drivers
kernel-versionnum.src.rpm
Includes the source code for the RHEL kernel

How to compile a new kernel in Linux (Centos-5.3)



when we  purchased a new hardware but the current kernel does not support it, or need to add functionality to the kernel that does not come from the factory.

It is at those times that is necessary to compile a new kernel for the device to be recognized or a new functionality can be used by the software.

Step:1 Install the required packgaes  for compling the new kernel

# yum install gcc make bison ncurses-devel rpm-build

Step:2 Download the latest Kernel using this url “ http://www.kernel.org/pub/linux/kernel/ ” , in my case I am using linux-2.6.36.2.tar.bz2" package .

# bunzip2 linux-2.6.36.2.tar.bz2
# tar xvf linux-2.6.36.2.tar
Step:3 Now got to extracted directory of latest kernel

#cd linux-2.6.36.2
Copy Kernel source code directory "linux-2.6.36.2"in "/usr/src" directory,

#cp -r linux-2.6.36.2 /usr/src/

Step:4 Go to "/usr/src/linux-2.6.36.2/" and run, 
#make menuconfig



This will open Linux Kernel configuration window for you to select & load extra modules. Make selection as below, you can choose something different .

Step:5  Select modules whatever you want, Click "Enter" to go to sub menu, & press "Space" to enable , load & disable modules.


In my case i am enabling NTFS file system in Linux, to make it enable do like this





Disk Partitioning & Managing Partitions

Disk Partitioning & Managing Partitions

When we format a computer hard drive, we will lose everything that is on the drive. Therefore, it is very important to back up anything you might later want.To format a secondary drive, we  need root access. Linux allows only 4 primary partitions.

On an IDE drive, the first drive is called hda, and the partitions are shown as hda1, hda2 . . . . etc. etc. Your second drive is called hdb.On an IDE drive you can have up to 63 partitions, 3 primary and 60 logical ( contained in one extended partition )

On a SCSI drive, the first drive is called sda, the partitions are sda1, sda2 . . The second drive is called sdb.On an IDE drive you can have up to 63 partitions, 3 primary and 60 logical ( contained in one extended partition )
An extended partition is the only kind of partition that  can have multiple partitions inside. Think of it like a box that contains other boxes, the logical partitions. The extended partition can't store anything, it's just a holder for logical partitions.

The extended partitions is a way to get around the fact you can only have four primary partitions on a drive. You can put lots of logical partitions inside it.

We can see all the drives attached to your system by typing the command "ls /dev/hd*" or "ls /dev/sd*", depending on which type (IDE, SATA and so forth) the drives are. On the example system, the result of this command looks like "/dev/hda /dev/hda1 /dev/hda2 /dev/hdb /dev/hdb1". The operating system is installed on hda, which has two partitions (hda1 and hda2), and there is one partition on hdb and hdb1.

Steps for Creating partition using fdisk command : -

Step:1  To list available drives on you machine type:
# fdisk -l
Choose the drive you want to make changes to and engage it using fdisk:
# fdisk /dev/sdc
replace the "sdc" with the drive you want to edit.

The basic fdisk commands you need are:
m - print help
p - print the partition table
n - create a new partition
d - delete a partition
q - quit without saving changes
w - write the new partition table and exit

Step:2  Enter "p" to see the partition table of the drive. The first line of output from the "p" command will also tell you the size of the drive. This is a good way to double-check that you are working with the correct drive.

Step:3 Type "n" and hit "Enter." Then press "p" to create a primary partition. It asks you for a partition number; enter "1." Now you are asked which cylinder the partition should start at. The beginning of the drive is the default, so just hit "Enter." Then, you are asked for the last cylinder. The end of the drive is the default, so you can just press "Enter" again.


Step:4 Now you are back at fdisk's command prompt. Use the "p" command to check the partition table. You should now see your new partition at the bottom of the output.


Step:5   Now  we need to set the filesystem type for your new partition with the "t" command. We are asked for the Hex code of the filesystem you wish to use. We will use the standard Linux ext2 filesystem, which is "83." If you are doing something special and know of a particular filesystem that you need to use, you can press "L" to see all the codes, which are one or two characters made up of the numbers 0 to 9 and the letters a to f.

Process and Daemon Process in Linux

Process :-  Process can be defined as “ Any program in execution is called processs”

Daemon Process :- A daemon process is a program that is run in the background and provide some system services. On a Linux system, some processes start at boot time and continue running until the system is powered down or until you execute a kill command. These processes typically provide some specific function such as serving pages , sharing print queue or processing loging requests. In Unix World these types of processes are called daemons.

A daemon process runs in the background , waiting for events to happen , When the daemon receives a request , typically it forks a copy of itself to process the request. Meanwhile the parent goes back to wait for the next event or request.
The Apache Webserver is an example of a daemon, It starts running when the system first boots, and remains in the memory servicing requests for web pages until the system is shutdown or “httpd”
daemon is manually stopped.

Types of Daemons :-
  1. Standalone
  2. Transient (Controlled by the super daemon xinetd)
Daemons may be started up in a number of ways. “standalone” daemons are programs that run all the time & are started up either bu init (such as mingetty and gdm) or by a startup script under /etc/rc.d. “Transient” daemons are only started up when they are needed,& are controlled by a standalone daemon called xinetd. Standalone daemons can be used more quickly, but take up the space in the memory and cpu cycles all the time, whether or not they are actively being used.

Typically a daemon  process is not connected to a TTY, so is shown in a ps report with a TTY of '?'.From this , you may also recognize processes which are managed by daemons, as they  too will show a '?' in the TTY field

Process States :-

Running: This is a state where a process is either in running or ready to run.
Interruptible: This state is a blocked state of a process which awaits for an event or a signal from another process
Uninterruptible: It is also a blocked state. The process is forced to halt for certain condition that a hardware status is waited and a signal could not be handled.
Stopped: Once the process is completed, this state occurs. This process can be restarted
Zombie or  Defunct process : Just before a process dies ,it notifies its parent & waits for acknowledgement .If the parent process does not acknowledge this notification,all the resources except for the PID are released . Zombie Process are cleared from the system during the next system reboot.
Command to see the zombie process :

[root@mail7 ~]# ps -ef

UID

PID

PPID


STIME

TTY

TIME

CMD
root100May10?00:00:00init [3]
root210May10?00:00:12[migration/0]
root310May10?00:00:00[ksoftirqd/0]
root410May10?00:00:00[watchdog/0]
root510May10?00:00:00[events/0]

Where:
  • -e to display all the processes.
  • -f to display full format listing.