{"id":270,"date":"2012-08-14T11:14:18","date_gmt":"2012-08-14T15:14:18","guid":{"rendered":"http:\/\/sites.hampshire.edu\/ci-lab\/?p=270"},"modified":"2014-03-31T16:11:18","modified_gmt":"2014-03-31T20:11:18","slug":"how-to-use-single-nodes-of-fly-for-clojush-runs-2","status":"publish","type":"post","link":"https:\/\/sites.hampshire.edu\/ci-lab\/2012\/08\/14\/how-to-use-single-nodes-of-fly-for-clojush-runs-2\/","title":{"rendered":"How to use single nodes of fly for Clojush runs"},"content":{"rendered":"

0. This is ancient. You probably don’t want to do it this way. See this<\/a> and this<\/a> and this<\/a>.<\/h3>\n

1. Set up a simple Clojush project using Clooj and Leiningen on a mac:<\/h3>\n

In Clooj: create a project called clusterdemo with namespace clusterdemo.core<\/p>\n

Edit the project’s project.clj to include a dependency for [local-file “0.0.4”]. The one that I’m doing this with right now is a Clojure 1.2 project and my edited project.clj file looks like this:<\/p>\n

(defproject clusterdemo \"1.0.0-SNAPSHOT\"\r\n\u00a0:description \"FIXME: write\"\r\n\u00a0:dependencies [[org.clojure\/clojure \"1.2.1\"]\r\n[org.clojure\/clojure-contrib \"1.2.0\"]\r\n[local-file \"0.0.4\"]])<\/pre>\n
In Terminal: cd into the project directory and run “lein deps”<\/p>\n
In Finder or Terminal: put a copy of clojush.clj into the project’s src directory<\/p>\n
Relaunch Clooj (necessary for Clooj to see all of the files added above).<\/p>\n
Open core.clj and add code for a pushgp run; here’s a very simple thing to get things going:<\/p>\n
(ns clusterdemo.core\r\n(:use [clojush]))\r\n(pushgp\r\n\u00a0:error-function (fn [program]\r\n(let [top-int (top-item\u00a0:integer\r\n(run-push program (make-push-state)))]\r\n(if (number? top-int)\r\n[(Math\/abs (- top-int 10))]\r\n[100])))\r\n\u00a0:atom-generators (list 1 'integer_add))<\/pre>\n
In Clooj: do REPL > “Evaluate entire file” to make sure that it works.<\/p>\n
2. Set up a directory on the cluster for your run:<\/h3>\n
Lees-MacBook-Pro:clusterdemo leespector$ ssh -l lspector fly.hampshire.edu\r\nlspector@fly.hampshire.edu's password:\r\nLast login: Sun Mar 11 19:11:45 2012 from c-71-192-29-61.hsd1.ma.comcast.net\r\n[etc]<\/pre>\n
We’re going to ssh to compute-1-1 and use it as a single computer.<\/p>\n
[lspector@fly ~]$ ssh compute-1-1\r\nRocks Compute Node\r\n[etc]\r\n\r\n[lspector@compute-1-1 ~]$ mkdir clusterdemo\r\n\r\n[lspector@compute-1-1 ~]$ logout\r\nConnection to compute-1-1 closed.\r\n\r\n[lspector@fly ~]$ logout\r\nConnection to fly.hampshire.edu closed.<\/pre>\n
We’ve now set up a folder for our runs (which is still empty). Note that the folder is actually on all nodes — your user directory is cross-mounted on all nodes. So you didn’t really have to ssh to to compute-1-1 before making the directory, because whether you’re on the head node or sshed to any node you’ll still be (initially, on login) in the same directory. But I sshed to compute-1-1 to make it feel more like I was going to that computer. Note that if I wanted to have another run going at the same time on another node that I should make a separate directory for that (e.g. clusterdemo2 for a run on compute-1-2) and repeat everything here for populating that other directory and starting that other run.<\/p>\n
Here’s what’s in my project directory on my mac now:<\/p>\n
Lees-MacBook-Pro:clusterdemo leespector$ ls\r\nclasses lib project.clj src\r\n\r\nLees-MacBook-Pro:clusterdemo leespector$ ls lib\r\nclojure-1.2.1.jar local-file-0.0.4.jar\r\nclojure-contrib-1.2.0.jar\r\n\r\nLees-MacBook-Pro:clusterdemo leespector$ ls src\r\nclojush.clj clusterdemo\r\n\r\nLees-MacBook-Pro:clusterdemo leespector$ ls src\/clusterdemo\/\r\ncore.clj<\/pre>\n
Let’s move all of that to the clusterdemo directory on the cluster, but we’ll just put all of the files in the same top-level directory and not worry about recreating the subdirectories there:<\/p>\n
Lees-MacBook-Pro:clusterdemo leespector$ scp lib\/* lspector@fly.hampshire.edu:clusterdemo\/\r\nlspector@fly.hampshire.edu's password:\r\nclojure-1.2.1.jar 100% 3165KB 1.0MB\/s 00:03\r\nclojure-contrib-1.2.0.jar 100% 466KB 465.9KB\/s 00:00\r\nlocal-file-0.0.4.jar 100% 2290 2.2KB\/s 00:00\r\n\r\nLees-MacBook-Pro:clusterdemo leespector$ scp src\/clojush.clj lspector@fly.hampshire.edu:clusterdemo\/\r\nlspector@fly.hampshire.edu's password:\r\nclojush.clj 100% 93KB 92.6KB\/s 00:00\r\n\r\nLees-MacBook-Pro:clusterdemo leespector$ scp src\/clusterdemo\/core.clj lspector@fly.hampshire.edu:clusterdemo\/\r\nlspector@fly.hampshire.edu's password:\r\ncore.clj 100% 390 0.4KB\/s 00:00<\/pre>\n
3. Connect to compute-1-1 again and conduct a run:<\/h3>\n
Lees-MacBook-Pro:clusterdemo leespector$ ssh -l lspector fly.hampshire.edu\r\nlspector@fly.hampshire.edu's password:\r\nLast login: Sun Mar 18 20:29:43 2012 from c-71-192-29-61.hsd1.ma.comcast.net\r\n[etc.]\r\n\r\n[lspector@fly ~]$ ssh compute-1-1\r\nLast login: Sun Mar 18 20:30:31 2012 from fly.local\r\n[etc.]\r\n\r\n[lspector@compute-1-1 ~]$ cd clusterdemo\r\n\r\n[lspector@compute-1-1 clusterdemo]$ ls\r\nclojure-1.2.1.jar clojush.clj local-file-0.0.4.jar\r\nclojure-contrib-1.2.0.jar core.clj<\/pre>\n
Let’s make a “command” run script that sets up the java classpath correctly (which is hard to remember), starts a run, and sends output both to the screen and to a file called “out”:<\/p>\n
[lspector@compute-1-1 clusterdemo]$ echo \"java -cp $PWD:.\/*: clojure.main -i core.clj | tee out\" > command\r\n\r\n[lspector@compute-1-1 clusterdemo]$ chmod +x command<\/pre>\n
Run it:<\/p>\n
[lspector@compute-1-1 clusterdemo]$ .\/command<\/pre>\n
That should spew all of the output from a run, which should succeed quickly and leave you hanging after the program has been simplified.<\/p>\n
Use Cntrl-C to exit.<\/p>\n
Check the “out” file to see that it contains the output.<\/p>\n
4. Conduct long and\/or multiple simultaneous runs:<\/h3>\n
If you can keep your terminal session open for an entire run then you don’t need to do any more than what has been described above (but with a different problem file, etc.). If you want to conduct multiple runs on multiple nodes simultaneously then just open a new Terminal window for each, make a directory on your fly account for each, ssh to a different node for launching each, and cd into the node-appropriate directory before launching each run.<\/p>\n
If you can’t keep your terminal session open (e.g. because you have to move your computer) then conduct your runs within “screen”. To do this, when you are connected to the node type “screen” and return to enter a screen session; then start your run and while it’s running type Cntrl-A-D to disconnect from the screen. You will stop seeing the output but process will keep running within screen. Then you can logout or whatever, and when you come back you can type “screen -r” to resume your screen session. When you’re totally done and you want to terminate your screen session you can do that with Cntrl-D.<\/p>\n","protected":false},"excerpt":{"rendered":"
0. This is ancient. You probably don’t want to do it this way. See this and this and this. 1. Set up a simple Clojush project using Clooj and Leiningen on a mac: In Clooj: create a project called clusterdemo with namespace clusterdemo.core Edit the project’s project.clj to include a dependency for [local-file “0.0.4”]. The […]<\/p>\n","protected":false},"author":622,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-270","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/posts\/270"}],"collection":[{"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/users\/622"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/comments?post=270"}],"version-history":[{"count":3,"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/posts\/270\/revisions"}],"predecessor-version":[{"id":478,"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/posts\/270\/revisions\/478"}],"wp:attachment":[{"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/media?parent=270"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/categories?post=270"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.hampshire.edu\/ci-lab\/wp-json\/wp\/v2\/tags?post=270"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}