BOINCpe: Advanced Configuration

BOINCpe-LogoThe steps explained for basic setup enable any computer to be a functional BOINC crunching machine very quickly. However, there are still some things that have to be manually adjusted with this very basic configuration.

The more advanced configuration explained on this page will enable you to run your BOINCpe machines without any user interaction at all (beyond pressing the power button of the computer). You will be able to operate your machines diskless (without hard disk drive) and headless (without mouse, keyboard and monitor attached).

Configure BOINC Remote Access

Since you most probably want to monitor and configure the BOINC client running on the crunching machine you will have to configure remote access to this machine. The BOINC client already comes with RPC (Remote Procedure Call) capabilities. Essentially, you can use the BOINC manager GUI running on a different machine in order to connect to any other remote host (this case: BOINCpe).

The following steps only briefly describe how to enable Remote Access. Please have a look at the article „How To Control BOINC On Remote Computers“ from the BOINC Wiki. There are two text files necessary to configure remote access.

Create file remote_hosts.cfg in .\plugin\BOINC_client\data

You might have to create the file remote_hosts.cfg if you do not already have one. Actually, this file is a list of DNS names or IP addresses. Each one in a separate line. No separating characters are necessary except for the line breaks. Only the computers specified in this file will be allowed to access BOINC remotely.
See: Remote Hosts File

Edit/Create file gui_rpc_auth.cfg in .\plugin\BOINC_client\data

The file gui_rpc_auth.cfg configures the password necessary for accessing the client. It is a plain text file containing only a single line which is the password. This file is already available in directory .\plugin\BOINC_client\data. You might want to change the initial password, but you can also leave it as is.
See: BOINC Daemon Password

After putting these two files in the directory .\plugin\BOINC_client\data and building BOINCpe, your machine will be accessable by remote. If you are running multiple crunching machines in a farm, the free tool BoincView can support you in monitoring and administrating multiple BOINC machines at once…

Automatically get settings from a BOINC Account Manager (BAM)

A BOINC Account Manager (e.g. BOINCstats BAM!) is a great thing to manage your BOINC settings centrally across a number of different projects you might be attached to. You can configure BOINCpe to use your BAM account in order to start computation immediately after startup without further configuration. BOINC will then use your default BAM preferences.

You need access to the installation directory of BOINC that is already successfully connected to your BAM Account. From there you have to get the following files:

Copy file acct_mgr_url.xml to .\plugin\BOINC_client\data

This file identifies the account manager to be used (e.g. BOINCstats BAM!). Just take it from your BOINC installation directory and copy it to .\plugin\BOINC_client\data.

Copy/Edit file acct_mgr_login.xml to .\plugin\BOINC_client\data

The file acct_mgr_login.xml has two different purposes: First, it stores your personal login data for the BAM account (i.e. login name and hashed password). And then, it stores the ID of the host that connected to BAM the last time using these credentials.
After copying the file to .\plugin\BOINC_client\data we have to modify the file in order to get a new unique host ID (CPID) for the BOINCpe machine and to let it connect to BAM immediately. This is easy: Simply edit the contents of the XML tags „<previous_host_cpid>“ and „<next_rpc_time>“ to „0“. It has to look like the following:

...
<previous_host_cpid>0</previous_host_cpid>
<next_rpc_time>0</next_rpc_time>
...

Please remember that the host ID will be gone (this will most likely also lead to multiple machines in your project profile(s)), once the BOINCpe machine is restarted unless you configure the BOINCpe Persistency Service (see below).

Automatically backup your data using BOINCpe Persistency Service

What does BOINCpe Persistency Service do?

BOINCpe Persistency Service: What does it look like?Are you worried about using a RAM disk for your long running computations? Want to make sure your projects never have duplicate entries for your machines again? Then BOINC Persistency Service is right for you!
It will periodically backup your BOINC data from the RAM disk to a network share on a different computer (e.g. your file server). Moreover, it will automatically restore the BOINC data from the share after bootup to start computation with the backupped settings, WUs and checkpoints. Please check the diagram on the left for further information on how the persistency service works. See the BOINCpe FAQ for further configuration of the cron scheduler, if you like.

The configuration of the BOINCpe persistency service will work fine with as many machines as you like. Simply specify the same network share for all BOINCpe machines. It will automatically create/read a subdirectory according to a MAC address of the computer.
So, do your configuration for BOINCpe once – burn it as often as you need it…

Edit your share and login data in .\plugin\BOINC_persistency\files\network_share_data.bat

You need to enter your network share and credentials for accessing it. Please follow the instructions in .\plugin\BOINC_persistency\files\network_share_data.bat.

Note: Your BOINCpe system’s MAC address will be added as a directory to path you specify in this file. This is necessary in order to distinguish between the BOINC data of different hosts.
An Example: Let us assume your first BOINCpe host has the MAC address 00-0A-0B-0C-0D-0E and your second host has the address 00-01-02-03-04-05. You can use the same path for both hosts (e.g. \\myServer\myShare in network_share_data.bat). The final path will automatically be set to: \\myServer\myShare\00-0A-0B-0C-0D-0E respectively \\myServer\myShare\00-0A-0B-0C-0D-0E.
Also have a look at the above diagram for further illustration…

Enable the plugin „BOINCpe Persistency Service“ in the plugin menu of PEbuilder.exe

The plugin is disabled by default. You will have to enable it manually before building BOINCpe.

Note: The local data put in folder .\plugin\BOINC_client\data) will be overridden if using persistency service and valid BOINC data exists on the share. On startup, the data from the share is loaded in order to start BOINC.

You can manually load or store BOINC data using Go/Programs/BOINC/Load BOINC Data from Share… respectively Go/Programs/BOINC/Persist BOINC Data to Share…

Note: Always shut down the machine using Go/Shut down/Shut down. All BOINC data will be persisted before shutting down the computer.

28 Gedanken zu “BOINCpe: Advanced Configuration

  1. Simon Zadra (KWSN - Chicken of Angnor)

    BoincPE is a pretty cool idea!
    Nice of you to save me the time; I was thinking about making a WinPE-based Mini-BOINC image myself.

    The Persistency-Service could use a little tuning, also a little more documentation inside the scripts.

    Great job, I’m testing it on a Pentium-D right now (using my own optimized apps, too).

    Regards,
    Simon.

  2. Thomas Walton

    Non server versions of windows limit the number of connections that can be established. The persistency service needs to disconnect the mapped drive to avoid limiting the number of BoincPE workstations which can use this service.

  3. Torben Artikel Autor

    @Tom: Have you experienced this problem yourself? There is a parameter available at the network share dialog for your folder called „User limit“. Setting this to „Maximum allowed“ should actually solve your problems…

    Moreover, I don’t think that only because a share is mapped, a connection is held open all the time. It should be closed after each file transfer. If I check my open sessions at the host serving the persisted data, there aren’t any (it’s 2003 though).
    You can check yourself. Open the run box and type compmgmt.msc. Then select Shared Folders/Sessions. Are there any remaining sessions of your BOINCpe hosts?

    Regards,
    Torben

  4. Thomas Walton

    Yes. I have unlimited set. WInXP peer-peer limit is 10:

    RE: http://support.microsoft.com/kb/328459/

    „Computers that run Windows NT Workstation 4.0, Windows 2000 Professional, and Windows XP Professional are licensed for a maximum of 10 concurrent client incoming sessions. Computers that run Windows XP Home Edition are licensed for a maximum of 5 concurrent client incoming sessions. All logical drive, logical printer, and transport level connections combined from a single computer are one session.

    I have 11 systems online now, 5 more to finish ‚building‘.

    Connections stay open even after the peer computer has been shutdown (they do timeout).

    This causes problems sharing printers (they count in the max 10 inbound peer connections).

    I have 8 BartPe installations running right now and each has an established connection to to the share per the management console.

    I first noticed this when I put online three Win98 machines–they 11th one got the 71 error code.

  5. Torben Artikel Autor

    OK, didn’t know that…

    However, did you check out these two articles:
    http://support.microsoft.com/kb/122920/
    and
    http://support.microsoft.com/kb/138365/

    You could check the value of „autodisconnect“ at HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters

    Maybe you can get the sessions to become automatically disconnected using the net config server /autodisconnect:x command and setting it to a low value. It’s worth a try…

    It might also be a problem that the hosts try to persist their data simultaneously. Is this a problem as well?

    Regards,
    Torben

  6. Thomas Walton

    Default timeout is 15 minutes. Using „config server /autodisconnect:1“ will help when starting all the machines up at the same time.

    Launch, load, and persist batch files need to detect the error and reattempt every ‚x‘ minute(s) until completed.

    The client side appears to do a automatic reconnect when the mapped drive is accessed (handles timeouts gracefully).

    I did a staggard start of 12 systems and then shut them all down at the same time. 10 Shut down, two received the error message. After the open connections dropped to 8, I again shut down the remaining two and they saved their data.

    Now that I am aware of this problem, I can adjust how I start and stop the machines to avoid loss of data.

    What I didn’t ‚completly‘ test is when BoincPE starts and the folder can’t be mapped, BOinc is started anyway, creating a ’new‘ installation which I’ll assume on shutdown will overwrite the previous session

    I don’t recommend permitting Boinc to launch when the persistency service is unable to establish a connection. Do you agree? This also means you shouldn’t persist the data–it’s best just to wait forever for the mapped drive.

    I monitor with Boincview and when a client fails to come online after the CD light stops flashing, I vnc into that machine to see what’s wrong.

    Keep up the good work!

  7. Torben Artikel Autor

    Mhmm, the new cron service in the next version will help you to configure different schedules for the automated persistency.
    Alternatively, you can simply set the system clock of the BOINCpe hosts to different times. No one will ever care and it is specific to the particular host.

    If the network share is not mapped and nothing is restored for whatever reason, it is also impossible that the data of the „transient“ session will overwrite the „old“ data.
    The host will run the „new“ installation. If you restart it sometime and it can read the „old“ data from the share again all you have to do is merging the two hosts in your BOINC project web interface.

    Regards,
    Torben

  8. Thomas Walton

    Does acct_mgr_login.xml in addition to setting ‚previous_host_cpid‘ to 0, should ’next_rpc_time‘ set to 0 to force a BAM update and ‚host_authenticator‘ also be set to 0?

    Forcing a BAM update allows using the „Attach new host by default“ to connect to projects automatically.

  9. Torben Artikel Autor

    In my opinion the BAM update is triggered anyway, if the host is not connected to any project.

    Setting „previous_host_cpid“ to 0 was sufficient for me in the past. Did you experience that the host did not initially sync with your BAM?

  10. Thomas Walton

    Yes it had no work and it didn’t connect to BAM, but that’s because the manager template I used was ‚fresh‘, just copied from a client which had contacted BAM about 30 minutes prior, so it wasn’t scheduled for another 23.5 hrs to make contact.

  11. Thomas Walton

    Suspended work units which haven’t reached a checkpoint will stay in memory ignoring the „NO“ setting of „Leave applications in memory while suspended?“. This makes sense as a dump from memory without a check point means loss of data, but with limited memory and multiple projects it’s possible to run out of ’swap‘ space causing work units to fail.

    A note might be added warning users that choose to run multiple projects that they ’should‘ have enough ram to keep each project in memory while suspended.

    I first noticed this with RieselSieve. Its first checkpoint is at 10% and Boinc suspended it to start another project. After that all the other work units failed with ’no swap space‘. Although there was enough ram for each project to run correctly, I didn’t have enough to have RieselSieve in memory when another project started.

  12. Torben Artikel Autor

    I’m a bit busy right now…

    However, I don’t understand why the setting should be ignored. Do you think it’s a bug of the BOINC client?

  13. Torben Artikel Autor

    Thanks for this further investigation. I’ll have an eye on the thread at the QMC boards since this is also interesting to me…

    Regards,
    Torben

  14. Ferry

    Does anyone know how you can add the proxy server in the config by default ?
    I’ve been searching the web but haven’t found anything ..

    any help would be appreciated

  15. Torben Artikel Autor

    Hi Ferry,

    proxy information is stored in client_state.xml respectively client_state_prev.xml. You could try copying your client_state_prev.xml to the data directory of BOINCpe and remove the unnecessary lines. Try to keep only the following in the file:

    <client_state>

    <proxy_info>

    <use_http_proxy/>
    <use_socks_proxy/>
    <socks_version>5</socks_version>
    <socks_server_name>proxy.yourdomain.corp</socks_server_name>
    <socks_server_port>8080</socks_server_port>
    <http_server_name>proxy.yourdomain.corp</http_server_name>
    <http_server_port>8080</http_server_port>
    <socks5_user_name></socks5_user_name>
    <socks5_user_passwd></socks5_user_passwd>
    <http_user_name></http_user_name>
    <http_user_passwd></http_user_passwd>

    </proxy_info>

    </client_state>

    Please let me know if this works.

    Regards,
    Torben

  16. Eric Weitz

    Is there a way to change the MAC address of the network card with BOINCpe? I can do so through the Control Panel with Windows XP and Vista, but wouldn’t know where to begin with BOINCpe.

  17. Torben Artikel Autor

    Hi Eric,

    do you mean the IP address? You can change it in the file .\plugin\penetcfg\penetcfg.ini before burning the CD (you find instructions in Step #1 here).

    Alternatively, you might use the PeNetCfg utility after booting up BOINCpe. You find it in the „Go“ menu. However, you will have to enter the new address everytime the machine restarts.

    Hope this is what you wanted.

    Regards,
    Torben

  18. Torben Artikel Autor

    Hi Brian,

    you probably want to know if you can use W2k instead of Win XP as OS for BOINCpe, right?
    Unfortunately, BartPE (on which BOINCpe is based) cannot use Win 2000. See the following extract of the BartPE FAQ:

    Why can’t Windows 2000/NT4 be used to build BartPE? Is there a reason for this?
    Yes, that kernel does not support the „/minint“ switch and therefore cannot boot from readonly media… Also the layout.inf does not contain required information.

    However, you can, of course, create an XP disk of BOINCpe on a W2k machine.

    Cheers,
    Torben

  19. Heflin

    Stupid Windows!
    Stupid Me!!

    The remote_hosts.cfg file was saved as remote_hosts.cfg.txt on the first CD I burned. I corrected that & reburned a CD but still could not get into remote host.

    Turn out that with Persistancy set, the remote_hosts.cfg.txt file of the server was being used instead of the remote_hosts.cfg on the CD.

    Now to get the wirelss NIC enabled and used instead of the motherboard ethernet….

  20. Heflin

    The shutdown.cmd needs to be modified so if network persist is not there, try ONCE to recreate, then shutdown/reboot anyway.

    Or add shutnow.cmd which shuts down even if network persist is not there.

    YeHa! I’m looking to add a few more diskless machines runing boincpe

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.