Skip to content

Commit baea39c

Browse files
committed
Update documentation for first release.
1 parent c594c97 commit baea39c

File tree

3 files changed

+28
-3
lines changed

3 files changed

+28
-3
lines changed

Changelog.md

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# 0.1.0 "One fish"
2+
3+
Initial release of lobster for opportunistic computing:
4+
5+
* Tested with several thousand cores running
6+
* Uses `parrot_run` to access the CMS environment where not available
7+
* Uses chirp for parallel stage-out
8+
* Can query DBS3, or run over local data
9+
* Can publish to DBS3

README.md

+18-2
Original file line numberDiff line numberDiff line change
@@ -91,14 +91,14 @@ memory, 4 Gb for disk).
9191
The following environment variables may be set to influence the environment
9292
of the job, and may need adjusting to run on the site:
9393
<dl>
94-
<dt>PARROT_PATH</dt>
94+
<dt>PARROT\_PATH</dt>
9595
<dd>The path of <code>parrot_run</code>. Default is to look
9696
<code>parrot_run</code> in the <code>PATH</code> environment variable
9797
where lobster is started and send this version to the worker. If another
9898
parrot binary is preferred, set this environment variable to the
9999
corresponding directory.</dd>
100100

101-
<dt>PARROT_DEBUG_FLAGS</dt>
101+
<dt>PARROT\_DEBUG\_FLAGS</dt>
102102
<dd>Which debug flags to use. Default are none. See the <code>parrot_run</code>
103103
help for more details.</dd>
104104
</dl>
@@ -129,6 +129,22 @@ above:
129129

130130
wget --no-check-certificate https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py -O - | python - --user
131131

132+
## Using chirp with hadoop
133+
134+
On earth, do something akin to the following commands on earth:
135+
136+
cd /var/tmp/
137+
cp -r /usr/lib/hadoop/ .
138+
cp /usr/lib64/libhdfs* hadoop/lib/
139+
env JAVA_HOME=/etc/alternatives/java_sdk/ HADOOP_HOME=$PWD/hadoop \
140+
/afs/nd.edu/user37/ccl/software/cctools-autobuild/bin/chirp_server \
141+
--root=hdfs://ndcms.crc.nd.edu:19000/
142+
143+
and note the port chirp is running on. Then add the follow line to your
144+
lobster configuration and you should be all set:
145+
146+
stageout server: "earth.crc.nd.edu:<your_port>"
147+
132148
## Running opportunistically
133149

134150
The CRC login nodes `opteron`, `newcell`, and `crcfe01` are connected to

setup.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
setup(
66
name='Lobster',
7-
version='0.0.2',
7+
version='1.0',
88
description='Opportunistic HEP computing tool',
99
author='Anna Woodard, Matthias Wolf',
1010
url='https://github.com/matz-e/lobster',

0 commit comments

Comments
 (0)