DRQUEUE
From CRAVEwiki
Rendering is a process of creating an image from a model using a computer program called render engine.This is a high computational task and thus requires high CPU resource so in order to facilitate rendering it is better to distribute the rendering work over the network so that idle CPU's can be utilized to get the job of rendering done faster.
Contents |
Distributed Render QUEUE
As the name suggests it is a facility for queueing up processes for computing over a distributed environment. A distributed environnment need not necessarily mean a multiprocessing system. In CRAVE, using the network facility for distributing computing processes is of an optimum value. A simple reason to this is that, in CRAVE, we are using DRQUEUE to render frames provided by Blender Scene Files (.blend) into images or videos.
The output result, which may be an image (jpeg, png, targa, bmp) or a video (jpeg-avi, raw-avi), is specified by the Blender scene file. The scene file contains all possible information about the objects and their relations and their properties.
A Mini Introduction to the Theory of DRQUEUE
The main entities for the working of drqueue is
- master
- slave
- drqman
The /tmp,/logs in the /var/lib/drqueue are the directories that are to be shared in the network and are accessed by all client,slave & master.The /tmp will hold the jobs and /logs is the place where the master and the slaves will enter the details of the jobs.
For the basic theory of DRQUEUE go here
Requirements
- An NFS Server for Linux Clients
- A SAMBA Server for Windows Clients
- Blender - 3D Animation Tool
- Yafray - Rendering Engine (Not Necessary)
- BASH - Bourne Again Shell
- csh C shell
DRQUEUE @ CRAVE
The CRAVE Lab features the following system-level profile
- All clients (slaves) running Ubuntu Dapper Drake 6.06 for amd64 systems.
* Hardware constitutes AMD64bit 3GHz processor, 1 GB DDR4 RAM, 80GB Samsung PATA HDD. * Network is an Ethernet 10 Mbps LAN.
- Server running the Ubuntu Dapper Drake 6.06 for amd64
* Intel XEON Core Duo system currently with a single 64-bit processor. * 1GB DDR4 RAM * 160GB Samsung SATA HDD
Server/Master Configuration setup
* sudo apt-get install libgtk2.0-dev libgtk2.0-0
- The NFS master will be running on the Server machine. The DRQUEUE render-farm daemon and client programs will be installed on the NFS partition. To install the NFS Server:
* sudo apt-get install nfs-kernel-server nfs-common * edit /etc/exports as follows
[#] /etc/exports: the access control list for filesystems which may be exported
[#] to NFS clients. See exports(5).
/var/lib/drqueue/tmp/ 10.0.0.222(sync,rw) 10.0.0.223(sync,rw) 10.0.1.221(sync,rw) 10.0.1.222(sync,rw) 10.0.1.223(sync,rw) 10.0.1.224(sync,rw) 10.0.1.225(sync,rw) 10.0.1.226(sync,rw) 10.0.1.227(sync,rw) 10.0.1.228(sync,rw) 10.0.1.229(sync,rw) 10.0.1.230(sync,rw) 10.0.1.231(sync,rw) 10.0.1.232(sync,rw) 10.0.1.233(sync,rw) 10.0.1.234(sync,rw) 10.0.1.235(sync,rw) 10.0.1.236(sync,rw) 10.0.1.237(sync,rw) 10.0.1.238(sync,rw) 10.0.1.239(sync,rw)
/var/lib/drqueue/logs/ 10.0.0.222(sync,rw) 10.0.0.223(sync,rw) 10.0.1.221(sync,rw) 10.0.1.222(sync,rw) 10.0.1.223(sync,rw) 10.0.1.224(sync,rw) 10.0.1.225(sync,rw) 10.0.1.226(sync,rw) 10.0.1.227(sync,rw) 10.0.1.228(sync,rw) 10.0.1.229(sync,rw) 10.0.1.230(sync,rw) 10.0.1.231(sync,rw) 10.0.1.232(sync,rw) 10.0.1.233(sync,rw) 10.0.1.234(sync,rw) 10.0.1.235(sync,rw) 10.0.1.236(sync,rw) 10.0.1.237(sync,rw) 10.0.1.238(sync,rw) 10.0.1.239(sync,rw)
(No line-feed or return-carriage in the above list)
Note:1.By doing this we are permitting the respective directories to be exported i.e. used by the hosts by the listed computers only.
2.These entries has to be made in /etc/exports of the master machine only.
- The files /etc/hosts.allow and /etc/hosts.deny will be edited accordingly.
[to check with security issues!]
THE COMPLETE INSTALLATION PROCEDURE
*Install nfs server as explained before.
*Then install drqueue as sudo apt-get install drqueue The installation of drqueue has to be done on all the machine that has to be used in render farm.
By default when you do
sudo apt-get install drqueue
drqueue gets installed in the /var/lib folder
and the default configuration settings are also searched by the drqman in the same folder.
*Then set environment variables (.bashrc for the particular user or /etc/bash.bashrc for the all the users) as export DRQUEUE_ROOT=/var/lib/drqueue
export DRQUEUE_MASTER=10.0.1.221 (ie the server/master machine's address)
*Change the permission of /tmp /logs of /var/lib/drqueue of the master to read and write as.
sudo chmod 777 -R /var/lib/drqueue/tmp
sudo chmod 777 -R /var/lib/drqueue/logs
this is done so that the drqman can access the config files and binaries to work properly.
*Now copy paste all the files in /var/lib/drqueue/etc/drqueue to /var/lib/drqueue/etc.
this is a bug in the script of drqman for which we have to do this change.
*After this edit the /var/lib/drqueue/etc/drqman.conf file of the master as
logs=/var/lib/drqueue/logs
tmp=/var/lib/drqueue/tmp
db=/var/lib/drqueue/db
*Then change the /var/lib/drqueue/etc/master.conf file of the master as
logs=/var/lib/drqueue/logs
tmp=/var/lib/drqueue/tmp
db=/var/lib/drqueue/db
bin=/var/lib/drqueue/bin
etc=/var/lib/drqueue/etc
*Now change the /var/lib/drqueue/etc/slave.conf file of all the slaves(the machines you are using for rendering) as
logs=/var/lib/drqueue/logs tmp=/var/lib/drqueue/tmp pool=test1:test2,test3
*After this make changes in the /etc/fstab files of all the slave machine as
10.0.1.221:/var/lib/drqueue/tmp /var/lib/drqueue/tmp nfs defaults 0 0 10.0.1.221:/var/lib/drqueue/logs /var/lib/drqueue/logs nfs defaults 0 0 note:1.This will mount the /var/lib/drqueue/tmp and /var/lib/drqueue/logs of the server or the master on all the machine when the machine boots. 2.The master has to be started first so that the directories may get mounted on other machines.If this can't be done then we have to personally mount these directories in the machine through terminal as sudo mount 10.0.1.221:/var/lib/drqueue/tmp /var/lib/drqueue/tmp and 10.0.1.221:/var/lib/drqueue/logs /var/lib/drqueue/logs.
*Now reboot all the machines.And the installation is complete....
RUNNING THE RENDER FARM
- This is the Best part !!!
- Run the Master from the master machine as
$/var/lib/drqueue/bin/master
- Then run the slave in all the slave machines as
$/var/lib/drqueue/bin/slave
- Then run drqman from the master as
$drqman
- The drqman will launch which is a graphical interface and will show all the slaves connected in the computers window
it will also show all the jobs that is being done right now.
- To give a new job to the drqman right click on the jobs window where you will get the menu from where you can choose new job and other operations on the current jobs
- When you click on new jobs a new window will open where you can set the frame size the priority and the script and the job name in the main info window and the types of job as blender and the target scene files in the Kind of job window.
- The kind of job window has a button "create script" which will create the script that will guide the slave machine in the rendering process.
- Once the job is submitted you can monitor the job progress,the load distribution and other details by watching the main window or right click on the main window and then clicking on details in the menu shown.
- Thus the rendering is accomplished....
FINAL TIPS
- If all goes right the output of the rendering will be saved in the /tmp of the drqueue folder and will be in the form of avi frames.
- Any other error can be found by looking at the logs file of the drqueue folder.
- Make sure that the scene files (.blend files)are given the required permission so that it could be used by all the machine.
- Keep the scene file in the /tmp folder of the drqueue so that all the machine can access it.
- Yafray rendering worked for us without special efforts.
- Even cross platform rendering is possible on which we are still working.
- We can have a special launch icon made on the desktop to start the slave and master.
- A special folder could be shared in which we can share the scene file(.blend file),so that the output and the scene files can be kept separate from each other.
- The output files are in frame format which has to be re-rendered to make a continues movie so a concept of block is used we are trying to find out more about this feature.
- Remember to install csh it gave us a headache when we tried to run the blender script which was a csh script.
