@shijitht

October 14, 2010

Building a GNU/Linux system from scratch and running it with UML

Filed under: GNU/Linux, Projects — Tags: , , , , — shijitht @ 12:23 am

Other than installing the system from the precompiled binaries, we could build it from scratch. The scratch means, building every element that makes our system, entirely from source code !!!. This is a !!! now, but in older days this was a necessary installation routine for linux lovers. Since we are new to this process proper documentation is needed to build the system. A fantastic Linux From Scratch(LFS) project is there to help us. Go to http://www.linuxfromscratch.org/. With proper description of each step, this project is the best to start for a beginner.

After system build, we can use it on a physical drive as per the documentation. Compile a proper kernel and write it on a drive to boot from it. But this article covers using uml. UML means User Mode Linux. Run linux on top of current linux kernel instead of bare hardware. We can test the new system, above a uml kernel.  While running a uml kernel, we can specify virtual resources such as root file system, swap, hardware etc.

Why build from source ?

You will have a question that, why build from source when a lot distros are available, which could be installed easily with a few mouse clicks ?. The aim of installing from scratch is to make a proper understanding of our system software internals. You can see the use of each package, its contents, size and the commands and scripts to build it. With intelligent package managers, we are unaware of the horrible dependency relations between packages. When you finish compiling the system, you can also get an awareness about the effort people put in making it.

UML

Uml kernel runs on top of system kernel in user space. Being a user process, privileged direct hardware access cant be made in uml kernel. So these are converted to system’s native calls. These calls are usually in arch folder of kernel. This make the creation of UML kernel easier because only the arch section differs. Only that folder needs rewrite. Another simple technique is used to execute processes inside uml system. An executing process is traced using ptrace. On a system call the process is stopped and it’s called address is replaced by the address of system call in real kernel. Then the stopped process is continued.

Building UML kernel

Follow these steps to create a uml kernel binary.
Download a stable kernel source from kernel.org.
Untar and cd to that folder.
Give these commands in the command line.
# make defconfig ARCH=um
will produce a .config file with default configuration for uml.
# make menuconfig
if desired
# make menuconfig ARCH=um
to produce a custom kernel
# make mrproper
# make mrproper ARCH=um
to get rid of all traces of whatever building you did, and start over.
# make ARCH=um
starts build
On completion, you can see a uml binary called “linux” of size ~25M.
Strip off debugging symbols to cut size.
To test this kernel start it with ubda=< FILE SYSTEM > option.

Creating file system

We can use a virtual file system to check the above kernel.
Use dd command to create a file of 6 GB.
# dd if=/dev/zero of=fs bs=1G count=6
will copy 1G from /dev/zero 6 times measuring a total of 6G.
Now create a file system of type ext3 using mkfs.ext3.
# mkfs.ext3 fs
give yes when prompted and it will produce a virtual file system of type ext3.
Use loopback to mount it
# mount -o loop fs /mnt/lfs
will mount fs -> /mnt/lfs
Loop back device /dev/loop0 is used for mounting this file. Through this device we can use the file as we are accessing a block device.
Next step is to make all programs inside fs which will help us use the system.

Creating files(programs)

Our system needs a lot of programs and tools for proper functioning. This include the init process, the first process which loads at boot. Then bash to interact with user and various tools to do tasks like compilation(gcc). The proper documentation for building is given at linuxfromscratch.org. So repeating them again will be a waste of time. Go to LFS site and follow the step by step instructions to build a working system. Do the version check as per the documentation to make sure you meet all the prerequisites. Also some errors might happen if texinfo packages is missing. You can come back, when you finish setting up system boot scripts.

Booting in uml

Make some changes to what you have done.
Edit /etc/fstab
When booting with uml we specify ubda as the root file system device.
So add it to fstab instead of what is there as root device.
/dev/ubda    /    ext3    defaults    1    1
Edit /dev/inittab
Comment all agetty lines and add this line instead
1:2345:respawn:/sbin/agetty tty0 9600
Now you are ready to boot with uml.

Power ON

Unmount the file system
# umount /mnt/lfs
Use the uml kernel produced(“linux”)  to start the system
# ./linux ubda=< PATH TO fs >
This will take you to a new virtual system build with your hands.

Conclusion

Building a Gnu/linux system from scratch is an experience which will take you to each and every corner of your system. Familiarizing with many commands and scripting techniques will increase knowledge. The role of each package is clearly understood while building from source. Now we know, what each package has and depends on. So later a much lighter system can be built with a custom configuration as per demands. Very small(~4M) systems can be built, which could be used in embedded systems. If we use uml, no restart is needed to test or boot a system. Also an exposure to virtualization can be gained with uml. LFS project has a documentation that any beginner can follow. Building a linux system from scratch is something that every linux enthusiast must do.

September 26, 2010

Vertex coloring problem app powered by Ruby On Rails

Filed under: Projects — Tags: , , , , — shijitht @ 8:39 pm

Rails is a powerful framework for developing dynamic web applications. It is written in Ruby. Ruby is a must to write application back-end. It follows Model View Controller(MVC) to organize application. This app is to solve vertex coloring problem using Rubyonrails.

The problem is, no two adjacent nodes in the graph should have the same color. The input is a graph and output its colored form. The output is shown in the browser using HTML 5 and JavaScript. Heroku provides free hosting for rails apps. So I also covered using git and heroku to deploy it on-line.

Ruby On Rails

Rails framework eases the creation of dynamic and rich web applications. It allows a developer to start from scratch or from inbuilt templates. Use scaffold to create various templates. Rails got a way of its own to develop applications. First we need to master it. If we wont read the manual, experiences with other frameworks and web development will lead us to trouble. Motto of rails to do more with very less effort.

MVC in rails

Splitting the application into MVC will isolate logic from view, code remains clean and will make it clear where different types of code belong for easier maintenance.

M – Model

Model represents the data and the rules to manipulate that data. Models are used for interaction with the database. i.e. One model for a table in our app. In rails interaction between application and database is achieved using active records.

Active records is an ORM(Object Relation Mapping) programming technique. ORM means mapping of objects between program and database for storage and processing. Actually rails uses an extension of active records. Unlike other mapping, we don’t have to specify database schema. Just edit a few lines in database configuration file like user, password, socket etc and also specify a migration file to make changes while updating. SQLite is the default database, but we can change it to MySql or whatever in need by changing the above two files.

V – View

It represents the user interface of application. They are HTML files with embedded ruby code .

C – Controllers

Controllers process the incoming requests from the web browser, communicate with models for data and pass that data to views for presentation.

Getting Started

Installation

  1. Install ruby from your distro repo
  2. gem from source tar
  3. install rails using gem.

Gem manages ruby packages and libraries. More detailed installation steps can be found at http://railstutorials.org/book/.

Setup

To start a new project

  1. $ rails new <project name>
  2. Use ‘bundle install’ to install all dependencies specified in Gemfile.
  3. $ rails generate controller home index
    This will create a basic template to start a project. To test it, start server.
  4. $ rails server
    starts server at http://localhost:3000. It shows a rail welcoming page.

To replace it with the new application index page follow these steps.

  • remove public/index.html
  • set default root route in /config/routes.rb
    i.e. Uncomment the root :to => line and change it to “home#index”

Development

The app folder holds the MVC files of the app. The controller have the controller file. It will have a default action, index. You can edit it to add new actions. To reflect addition of new actions, these should be added to to the routes.rb file. Each action can have methods. For each action, corresponding view should be placed in views folder. The URL is of the form http://localhost:3000/controller/action/method/.

Views folder hosts html files and layout. By default, /public/javascript/application.js and /public/stylesheet/application.css can be used to store js and css. To include a specific file, layout in views can be used. Give the file name in double quotes near stylesheet_link_tag and javascript_include_tag.

Params can be used to receive values posted using POST. eg: to receive a parameter with name area use params[ :area ] or params[ “area” ]. The processed values can be embedded to the html page using <%= and %> tags. The control or loop expressions are used inside <% and %> and to use a value of a variable use <%= and %>.

Deploying online

Heroku provides a free hosting for rails applications. To use it, create a heroku account from http://heroku.com/. For installation do,

  1. $ sudo gem install heroku
  2. Now move to your git folder for rail app.
  3. $ heroku create
    Give credentials when prompted and follow the link shown to go to your app page.
  4. $ git push heroku master
    This will upload app.

Now go to the previous link to test application online.

Conclusion

Rails is a very good framework to develop web applications in a shot time and with less effort. The MVC model keeps the design simple and database can be used without worrying about schema. In built template system gives more time for user specific development. And using git and heroku, application can be loaded online. To check my code and its online presentation go to:

Git – http://github.com/shijith/ColoringProblem-Rails/
Heroku – http://afternoon-wind-21.heroku.com/

September 20, 2010

Vertex coloring problem app in Google app engine

Filed under: Projects — shijitht @ 12:37 am

Google App Engine lets you create and deploy apps on the same machine which servers google apps like Gmail. As we all know, it is a cloud infrastructure. Sandbox is the name of the virtual environment(machine), for running our apps. Our aim is to create an app to solve graph(vertex) coloring problem.

The problem is, no two adjacent nodes in the graph should have the same color. The input is a graph and  output  its colored form. The output is shown in the browser and use of HTML 5, JavaScript and Django is gonna make it interesting.

Google App Engine

Goole App Engine allows your apps to run in the sandbox. Like said, it is a secure box. It got some restrictions like can’t write to filesystem, only read files uploaded by the app, use specified APIs to get access to resources and services and there is a dead line for responses. Violation of these limits will end in an exception. And the free account limits a user in other ways like the usual storage limit, connection limit. Go visit the site for more details.

The app engine now supports two runtime, python and java. I prefer python, so I gone a stick to it. The sandbox has a standard python interpreter. The libraries which conflicts with the above mentioned policies are removed. It provides a data model to store data and api for managing it. The web frameworks like Django are also supported. As the site has a more detailed documentation for theses things, lets move to what I have done.

Getting started

Google App Engine resides in a remote server. So there will be question about how to develop offline. The emulation of app engine can be produced offline using App Engine SDK. It is available online. Install it and conform the requirements like run time support are satisfied in your system.

Development

Start the project by creating a folder for it. The app consists of mainly three type of files. app.yaml, project.py and index.html. This is the case for a simple project. To bring order in a little bigger project, the css, js, image etc folders could be included and more html pages for better functionality. The app.yaml file should be updated to reflect these changes.

app.yaml is the configuration file of app engine. It specifies the runtime, version, url handling etc. The python script does all data processing and transactions. To provide an MVC, the html elements are separated from the script(model) and put in separate html files(view). The current situation gives a lot of web tools and frameworks to develop front end. And app engine supports most of the frameworks. The SDK comes with inbuilt webapp and Django support. I used a mix of HTML 5, JavaScript, CSS, Django as front-end and python, app engine as back-end.

Canvas element in HTML 5 provided the canvas and images were drawn with JavaScript. CSS styles the pages and Django allows an easy integration with server for the creation of dynamic pages. You can find the code at http://github.com/shijith/ColoringProblem. This app is hosted under http://coloringproblem.appspot.com. For an example, see an instance of coloring

Some ideas

While developing a clicking application like this, a certain doubt is; How could you know, whether the click was on a circle or not. The answer is, for a circle at (0,0)

x^2 + y^2 = r^2 .

Let the circle be at (x1,y1) and click at (x2,y2). A point inside the circle should satisfy,

(x2 - x1)^2 + (y2 - y1)^2 <= r^2 .

Conclusion

Google app engine provides a very fast and reliable server for our apps and at no cost. HTML 5 includes a lot of in build elements for the creation of heavy graphics applications with JavaScript. The recent developments in browser technology fastened (300X) JavaScript execution and hence providing for the overall performance. With CSS 3 more effects can be added with less effort like round curves or a box. App engine dashboard provides detailed error report which eases error detection and debugging.

Create a free website or blog at WordPress.com.