Present: Lee, Tom (scribe), Nic, Mike, Tim (hangouts)

GECCO Submissions

• Nic: Xover bias
• Tom: Benchmarks
• Bill: EHC

GPTP

• Talk about who might be attending
• What should we write up?
  • Maybe simplification as post-run generalization improver?
    • Maybe in trees?
    • Maybe non-hill climbing using silencing?

Pucks

• Binding
• Involuntary transactins

Finite Algebras

Lee reimplemented

Present: Lee, Nic, Lance, Tom (scribe), Ben, Beryl, Tim, Wren, Eddie

Finishing some Student Project Ideas from Last Week

Eddie – He’s tried some stuff with Brevis and Quill. He might next work some with Pucks.

Ben – Not sure yet. Maybe something A-Life-y?

Tom’s Benchmark Problems

Should we include problems beyond iJava and Yuriy’s paper?

• We should have a citable source outside our work for the problem — i.e. not “we made it up”. Otherwise, probably “yes”.
  • FizzBuzz (kata) – yes
  • Bowling (kata) – maybe
  • Factorial – maybe (too easy?)
  • WC – yes
  • US Change – maybe (need citable source)
• Tom will consult rest of committee.

Pollen Tubes

Wren is going to try some things and consult others to try to see if a solution already exists.

Homology of Population

• Levenshtein distance of pairs of programs seems like good measurement.
• Likely can’t test all pairs, since this will usually be about 1 million Levenshtein distance computations. This will probably be too slow.
• Instead, sample random pairs of individuals — maybe on the order of 1000 to 10000.
  • May need to do validation to make sure the sample results really reflect the population.
  • Report: median, quartiles, mean, standard deviation.

Nic McPhee Presents: Analysis of Ancestry in GP with a Graph DB

• Using Neo4j – a graph database – to store ancestry info throughout a GP run.
• Makes ancestry info easy to collect and query.
• NeoCons – Clojure with Neo4j
• Cool stats can be queried from the database!
• Could be neat to look at common ancestries using different parent selection methods!

Present: Lee, Tom, Nic, Lance, Mike, Tim, Bill, Kwaku (virtual), Karthik (virtual)

Tom’s Benchmarks

He’d gladly have help coding and doing experiments on program synthesis benchmark problems.

Clojush 2

Clojush 2 is now live. It adds a linear representation for genomes, which is translated into regular Push programs for execution. People should try it out and report bugs!

Documentation of Recent Push

Mike (and Nic) will be working on documenting the use of Push as they get to know it.

Bladder Cancer Data Set

Clojush 2 has the training error down near 70. Lee hasn’t figured out how to control overfitting yet.

Discussion of Paper: Evolving Neural Networks That Are Both Modular and Regular: HyperNeat Plus the Connection Cost Technique Categories and Subject Descriptors

• There are different kinds of modularity, and what we want is different from what they present.
• Evolving an interface for functions is hard!
  • Push environments might be helpful here.
• Varying the “environment” of the problem might help.
  • Lexicase is sort of like varying the environment. Maybe?
• It seems like it would be reasonable to add a test case that tries to improve modularity based on the compressibility of the program.
  • We’d likely be interested in minimizing the amount a program is compressed. Something like: (compressed size of program)/(uncompressed size of program)

Present: Lee, Tom (scribe), Tim

NO MEETING NEXT WEEK!

For the following week: Read the paper Evolving Neural Networks That Are Both Modular and Regular: HyperNeat Plus the Connection Cost Technique Categories and Subject Descriptors, which will be sent to the email list by Tom.

Dartmouth Trip

• Lee touted lexicase as potentially good for their work
• We got a bladder cancer data set to try things with.
• Lee did some initial hackings, starting with simple trees, and is now trying Clojush
• Tim will be working on this.

WC + Tags

No difference between tags and no tags 🙁

Plush

It could be that instructions are no-oping near the beginning of programs, either because of a bug or just because the required stacks are empty. We could track the number of no-oping instructions to see if it increases throughout a run. It could be interesting to see what percent of instructions noop at each program position during each generation — you could have generations on the x-axis and program positions on the y-axis, and a heatmap showing what percent of instructions noop.

Administrative

• Present: Lee, Tom (scribe), Tim, Karthik (remote), Bill (remote)
• Tom will practice GECCO talk next week
• Frode should send us GECCO slides shortly

Plush genomes -> Push programs

• Do we want any Push program to be able to be represented by a Plush genome?
  • Lee: Yes!
  • Tom: Maybe?
• To do so, we have new instructions:
  • meta_open – opens new paren
  • meta_splice_backward – deleted previous paren pair

Hill-climbing vs. Genetic Operators

• e.g. simplification, epigenetics, etc.
• Is it useful to test and revert (using a fitness evaluation), or is just making the change and letting selection work it out just as good?
• Maybe Bill or Tom should experiment with this — EHC vs. using epigenetic genetic operators with more generations to make up for EHC’s extra evaluations.
• Simplest is to never do hill climbing, just have genetic operators.
• Decision: Implement a wrapper argument (like :make-reverting) that can be used in genetic operator pipelines that cases the next operator to be reverted if not better than its input program.

Issue of what to do with child programs that exceed size limits

Decision: implement argument with these options:

• return clone of one parent (default, as previously)
• return empty program
• return random program
• return one parent with some part(s) deleted

Present: Lee, Tom (scribe) Karthik, Bill, Omri

We should invite Ryan Shams to a lab meeting.

Lee’s A-Life-y thing

• Lazy can be annoying
• footnote – agent handling of errors in Clojush should get better
• Reducers might be useful

Bill’s Wind Turbine Results

• Lots of data
• Some promising results so far
• May try co-evolutionary methods

Closeline Linear Push

So far, it is unclear whether this is “The Future” of Push, or just another tool that could be selectively applied depending on the problem. If selective, it would be nice to have a switch in Clojush to turn it on and off. If not, then we should make a final version so others can use it.

Either way, eventually the code should be cleaned up. In particular, maybe all epigenetic lines should be stored in a map of them, so that we can have closelines, offlines, etc.

Less Than 2000 Problem

• There might be something interesting to look into in comparing the complexity of the data to the complexity of the evolved programs. If programs are more complex, then maybe they don’t generalize?
• General principle of “start small” might help somewhere here:
  • Small population
  • small number of test cases
  • small program sizes
• Offline -> epigenetics might be helpful for getting smaller programs of the right size.

Present: Lee, Tom (scribe), Frode, Karthik (virtual)

Notes and Lab Page

• Get Karthik up to speed for posting lab notes
• Update lab pages when possible: especially fly section

Simplification Paper – Journal Article

• Put together poster paper and Frode’s workshop paper

Laziness Can Get You Into Trouble

Lee’s Heisenbug memory leak

Tom’s Selection Comparisons Lexicase/Tourney

• Try graphing lexicase selections with total error on the x-axis instead of rank
• Maybe look at minimum error by test case vectors to see if they are similar or different between runs. Could (but not necessarily) show whether or not the runs are exploring similar or different search spaces.

Present: Lee, Tom (scribe), Bill, Charles Ofria

Note: Student travel grants for GECCO are now open for application.

Diversity

• Charles suggests using (average) Hamming distance of output vectors instead of behavioral diversity to measure the population diversity
  • This will do a better job of measuring the coverage of the phenotypic space (or “trait coverage”)
  • could just sample this, don’t have to compare every pair of individuals

Discussion of co-solvability paper

• Very related in motivation to lexicase
• lexicase is mostly different in that it can select individuals that are really bad at a few test cases as long as they are really good at others.
• Also, co-solvability requires binary test cases, without an obvious way to use on non-binary cases

Attending: Lee, Tom (scribe), Bill, Omri, Kartik (virtual), Kwaku (virtual)

Z3

Karthik – can probably connect Clojush and Z3 through Java API.

Lexicase Paper Rejection

• Some possible additions ideas:
  • Investigate how much of pop is let around after each step of lexicase
  • time runs for this paper
• Lexicase speedup idea: use a lexicase comparator, and then find optimum in population
  • To do this, would need a lexicase comparator that takes 2 individuals (or error vectors) as well as a test case ordering, and returns the individual with better initial cases based on the ordering
  • Then, you can use the comparator to find the “max” (or “min”) of the population – basically, use it with reduce to find best individual for that ordering
  • We should include this as a possibility in paper, but probably not implement it.
• Thoughts on niching/fitness sharing:
  • We should definitely enhance our discussion with related techniques, but not necessarily implement and compare with them
  • We should clarify how lexicase is different
  • If we find one option that is common and simple, we could try it out if we have time.

Attending: Lee, Omri, Frode, Tom (hangout, scribe), Kwaku (hangout), Karthik (hangout)

Business

• We should think about what to do for GPTP.
• No concrete plans for Moshe Sipper.

Omri Div III

Will involve QGame and using it for educational purposes.

Simplification

May or may not use WC data depending on space constraints.

Calc Thoughts

• Last gasp for trying to get Calc to work
• Tried, didn’t work:
  • Simplified instruction set
  • “behaviors” = “domains” test case generation
  • Put “old” test cases at front of lexicase order
• New things to try:
  • Two alignment deviations: most of the time use 0, but sometimes use large (like 100)
  • ULTRA tags by running one parent program, and then when run second, when it tags something instead ULTRA what it would tag with whatever is already tagged there.
  • Similarly, could take each parent and create its tagspace, and then re-create a program from the tagspace, lining up positions of tagging instructions for the same tag by padding with ‘ultra-padding, which will be removed after ULTRA.
  • Maybe try trivial geography with lexicase.