• New members wanted! Lee already emailed his advisees, but this does not include all people from past GP classes, nor people that changed advisees while Lee was on sabbatical. He is going to do that.
• NO meeting next week.
• Following week, meeting on Friday 1-3pm at Hampshire (this will be the regular time henceforth.
• Zeke has been evolving R programs. Programs seem to be memorizing, not coming up with meaningful data. He would like to see how some “solutions” do on test data to see if they have any predictive power.
• We all want GP web apps, through Clojure and/or Clojurescript. Zeke and Lee have been working on this, random errors and bugs. Still being worked out–oh wait, we fixed it during meeting, now it works.
• Omri should really get the Flea Market all set and working (what a thought)!
• Lee looked at Emma’s TSM code, hasn’t yet tried out the examples.
• Emma included input instructions for TSM. Discussed possibility of non-uniform TSM generation, with high amount of pairs and tag instructions.
• Lee proposed a COSMOS experiment with non-parametric data: COSMOS versus no COSMOS. The idea would be to show that no COMSOS will show “significant” results after some “large” number of runs, BUT after a “larger” number of runs, it will no longer show significance. Ideally COSMOS would not be fooled by this ephemeral “significant result” (stardust?).
• Tom has been doing Bowling runs. Nothing groundbreaking, progress happening.
• Lee brought up a very interesting idea about evolving rules for cellular automata given image data.
• For next meetings’ status report, everybody should also include a reading suggestion.
• We all decided to be BFFs (friendship bracelets pending)!

Admin

Agenda

Admin (10min)

Discuss readings

General: Sumit Gulwani’s Spreadsheet Data Manipulation Using Examples (in email)

Suggested Background: Version Space Algebra and its Application to Programming by Demonstration 

Additional reading: Martin Rinard’s Example Driven Program Synthesis for End-User Programming

Possible followup with Gulwani: Feedback generation for intelligent tutoring systems?

The Tozier files

Pushing Zeke to Push and Clojure

“Fast” lexicase selection

Quotes re GECCO?

Finite algebra draft

Clojush updates needing merging? (recognize-literal)

Meeting Notes

• Lee will be meeting with Hampshire PR person about GECCO article. Get him quotes about GECCO if you want!
• We have merged the changes to master. NOTE: Make sure you follow the proceedure for updating code – don’t update on GitHub, since things get messed up that way.
• Finite Algebras paper draft will be emailed to those interested in reading and making suggestions.
• Tom will be using the cluster next week for some kata bowling runs.
• Zeke was encouraged to use Clojure and Push, but will likely stick to what he knows.
• Tom will delete fast-lexicase-selection and push to GitHub. The old version will still be on GitHub in old commits, so we can get it back later if we want.

The Tozier Files

• There are some valuable things in the Bill Tozier emails, but also some things that aren’t really what we want to do.
• Things that may be valuable to persue:
  • Try shorter test cases with kata bowling
  • Monitoring of runs during evolution. Piano Roll plots or other finer details may be worthwhile, but other graphical investigations that only require things that are currently printed in logs may be just as good.
  • Think about why we are failing, and what to try next.
  • Cheating: Interesting topic. It may be useful to cheat at the beginning to see what you are up against, and take away later once you understand the problem better. Plus, cheating can be seen in different lights depending on your goals.
  • Maybe remove some instructions to make the search space smaller (flush, yank, shove, …)
  • Try writing a solution by hand.

Discussion of Gulwani paper

• This makes us think it may be worthwhile to implement a stronger string / pattern matching library in Push.
• The problems solved here may be inspirations for future “programming problems” for GP to solve.
• Our motivations for program synthesis are similar to theirs.
• As for the intelligent tutoring system, Lee will contact Sumit Gulwani to investigate further collaboration.

Admin

Lee would like everyone to send weekly status reports either to him personally or to the list. His preference would be for all emails to go to the list, but if there is something that is more appropriate to just a few people, that’s fine, too.

Common reading (2 Goodman papers on statistics)

Discussed misconceptions about p-values (esp. the comment at the bottom of p997). Zeke mentioned a talk that might be of interest. Tom was interested in how this related to GPBenchmarks. There are two cases that were brought up. The first was the case where we’re just interested in finding a solution. This would be for problems where we do not expect to find solutions in general or where we know that finding solutions is very hard. The second case is the one wherein we are comparing techniques and want to measure how one compares to the other. This would be the case where we expect to find a solution and are interested in wether one techniue offers better performance than the other. This ist he case where p-values adn typically used and owuld need to be reassessed in light of the expressed problems with p-values.

Note also this series of blog posts that may be of interest:

Criticism 1 of NHST: Good Tools for Individual Researchers are not Good Tools for Research

Criticism 2 of NHST: NHST Conflates Rare Events with Evidence Against the Null Hypothesis

Criticism 3 of NHST: Essential Information is Lost When Transforming 2D Data into a 1D Measure

Criticism 4 of NHST: No Mechanism for Producing Substantive Cumulative Knowledge

Here is also a more recent article from Steven Goodman that summarizes problems with p-values.

Possible applications to education / intelligent tutoring systems

After contacting Sumit Gulwani, Lee is looking for connections with automated test generation. Tom has suggested jumping off from the bug-fixing work and applying to this problem. Since there are a lot of projects already scheduled, Lee is looking for opportunities to expand or connect our current projects to this work.

Project updates and planning for tangible results

Zeke: Working ont he logical elements of his chemistry system, looking to use spectra rather than graphical representations. Having issues with the ordering on the molecules (“Atom 0 problem”). Omri’s suggestion is to use the heaviest atom as the starting point. The plan for August is for Zeke to hammer out the representation and start evolving molecules as soon as possible.

Omri: Working on a new Clojush interpreter with scoping options, including a scoping specification called “periscope.” No recent work on the flea market. Goal by the end of August is to have the flea market environment available as a experimental evolutionary system. Still having issues with pathological cases for essential vs. nonessential goods.

Kwaku: Currently working on modularity isses.. Taking the top 20 individuals and their modules and re-inserting these seccessful program’s modules in the population. Currently looking for good problems.

Tom: Doing work on lexicase selection and exploring extant greedy approaches and current work on production systems. Lee believes that a graph or lattice-like strucutre that’s pre-computed will make this computation fast. Also working on kata bowling; have found solutions for easy configurations; none for hard configurations. Some solutions have been found using lexicase.

Emma: Possible future meeting topic: tutorial on how to use the database.

Next week’s readings:

Spreadsheet Data Manipulation Using Examples (sent in email), Sumit Gulwani, William R. Harris, Rishabh Singh

Example-Driven Program Synthesis for End-User Programming (sent in email), Marin C. Rinard

Dimensions in Program Synthesis, Sumit Gulwani

Also, for the theoetical background consider this paper on version algebras.

Plan:

Admin

New group readings?

GEVA-clj

TSM

Automatic programming prior work and direction; Microsoft work and generative testing (from Programming Clojure)

Notes:

**Administrative**

Lee’s not yet “dug out” from being away…

we’re meeting next thursday @ the regular time.

WE WILL BE MEEINT @ EMMA’S PLACE IN NORTHAMPTON 89 MARKET ST

finalizations will come electronically

talked about some funding stuff and potential quantum computing grant

**Regular meeting**

we’re talking about p values, statistics & COSMOS – emma

we should pick a reading for our next meeting or meetings… -interested in microsoft automatic programming -emmas stats stuff -generative testing from within book chapter

THIS NEEDS DISCUSSION FROM LIST

–Talked about grammar based stuff from irland geva – cla

–tag space machine –emma’s going to build it.

Admin

• GECCO receipts – get to Lee with signed forms

GECCO thoughts

• Our talks were well-received
• Maarten Keijzer ideas: Push Fourth (one stack), Bad Hash Function (tag based on content)
• GP best paper may be related to our work
• Discussion of GP as optimization vs. program induction
• Discussion of automatic software repair and Simon Harding’s multi-type Cartesian genetic programming.
• Emma: could see GP as a front-facing part of other hybrid ML algorithms, or as an exploratory technique to understand data better, as opposed to a last-ditch effort when other things don’t work.

Steady-state GP

• Lee gave a description
• We may be interested in trying out steady-state PushGP
• Also, not sure of relation to ALPS (Age-layered population structure)

Rewarding Tag Usage

• Kwaku and Emma plan to investigate rewarding tags
• Encouraging modularity explicitly in the fitness function may be useful
• Difficult to know what to reward
• Other tag ideas include a shared tag-space in the whole population, or hereditary tag spaces

Blog

• Emma wants to start a blog to encourage communication with the rest of the EC community about issues we think are important
• May concentrate on understanding problems

Admistritavia

• Matlab: We now have access to 3 Matlab licesnses and various modules. The Simulink module only has 2 licenses available and is to be used for Lee’s collaboration with Prof Kourosh Danai. If you are interested in having access to Matlab, email Lee directly for instructions on the install. IMPORTANT: when using Matlab, you will need to explicity release your license (i.e. do not just put your computer to sleep).
• Josiah would like to see someone become an expert in R.
• Next lab meeting: Friday, July 13th at 1:30 in our usual room.

Discuss paper : Benchmarks

Main issues: privledging of MBF and ignoring outliers, few programming problems suggested, no dynamic problems (Lee and Tom)

Suggestion for a non-trivial programming task: evolving CPS expressions for automatic compiler generation (not terribly compelling, but an example of a programming task that’s hard for humans but easy for machines)

This paper is discussed in the GP-1 session on Monday.

GECCO

(*) Lee would like someone to consider attending Stephanie Forrest’s Evolutionary Software Repair Tutorial that occurs during the second half (i.e. after Lee and Emma have presented) of UP workshop

(**) Suggestions for this time frame: Evolutionary Computation Software Systems Workshop (two blocks), Generative and Developmental Systems (latter block).

(***) Consider attending the IGEC/SS/SBSE Best Papers session or the GBML 4: Reinforcement Learning

(+) Consider attending GA 2: Dynamic Optimization & Exploration Techniques

(++) Consider attending Alife 2 Best Papers or GP 4 Best Papers

(+++) Consider attending GP 5: Empirical studies, which features Krzysztof Krawiec’s paper.

The poster session will be Tuesday evening.

Everyone should atted the SIGEVO meeting Wednesday morning (and also the two keynotes on previous days). Individual tracks will have their own keynote speakers. Lee will be attending Stuart Kaufman’s talk.

Everyone should at least read the first sentence of all abstracts from the proceedings when they get them!

Consider attending the best paper sessions.

Bill Tozier will be around throughout; Lee and Tom to chat with him.

SIGEVO meeting Monday evening for Lee.

Goals: network and don’t be shy about introducing yourself to someone.

• Omri gave his status update
• Tom refactored Clojush and everyone was happy
• Decision: After GECCO, we will meet on Friday (not Thursday), July 13th at 1:30PM
• We discussed whether we might discuss papers related to the Humies
• Decision: Omri will select a few candidates for a paper to read from the GECCO papers (this reading will be for the upcoming meeting on July 5th)
• We talked about Clojush ECJ integration
• Decision: Every member should set and make known their goals for the rest of the summer
• We talked about the Nimrod problem. Lee presented interesting “binary list recursive concatenation idea” (I just made that title up).
• We talked about making tags floats in Clojush, but decided it was not necessary at this time

Administrativia (including agenda/notes management)

Discussion of Krawiec’s GECCO 2012 paper

Discussion of items raised in status reports, possibly including:

• scheduling large numbers of runs on fly
• improving the speed of lexicase selection
• description of the kata bowling problem and its significance
• new formulation of the Nimrod problem
• rational tag machines (see https://hampedia.org/w/images/c/c2/Rtm.pdf)
• Omri’s associative map idea?

NOTES FROM MEETING:

we talked about the paper.

group todo item: talk to krzysztof at conference

minimal tag machine (Lee)

•  evolve a tag space
•  pairs & lists
•  people welcome to work on this on its own thing or as part of push.

Bowling problem

• introduction to problem
• Tom is still working on it
• maybe problem is a translation problem… z means 1, etc
• maps

talked about lex case fitness selection
-talked a lot about how long different ways would take to run

next week:
-nimrod

1:30-1:40 — Administrative Details

This is where matters that can be concluded with a quick conversation should be taken care of. If there are any quick one-on-one discussions that can be done during this time, they should happen here. Students not engaged directly in these conversations can be helping Omri get set up for his practice talk.

1:40-2:00 — Omri Practice Talk

2:00-2:15 — Feedback on Omri’s talk

2:15-2:45 — Paper discussion – Altenberg (For next week, read Krzysztof (Chris) Krawiec’s paper)

2:45-3:30 — Misc

• Discuss collaborative work space
• Discuss Clojush refactorization