Admin

Lee would like everyone to send weekly status reports either to him personally or to the list. His preference would be for all emails to go to the list, but if there is something that is more appropriate to just a few people, that’s fine, too.

Common reading (2 Goodman papers on statistics)

Discussed misconceptions about p-values (esp. the comment at the bottom of p997). Zeke mentioned a talk that might be of interest. Tom was interested in how this related to GPBenchmarks. There are two cases that were brought up. The first was the case where we’re just interested in finding a solution. This would be for problems where we do not expect to find solutions in general or where we know that finding solutions is very hard. The second case is the one wherein we are comparing techniques and want to measure how one compares to the other. This would be the case where we expect to find a solution and are interested in wether one techniue offers better performance than the other. This ist he case where p-values adn typically used and owuld need to be reassessed in light of the expressed problems with p-values.

Note also this series of blog posts that may be of interest:

Criticism 1 of NHST: Good Tools for Individual Researchers are not Good Tools for Research

Criticism 2 of NHST: NHST Conflates Rare Events with Evidence Against the Null Hypothesis

Criticism 3 of NHST: Essential Information is Lost When Transforming 2D Data into a 1D Measure

Criticism 4 of NHST: No Mechanism for Producing Substantive Cumulative Knowledge

Here is also a more recent article from Steven Goodman that summarizes problems with p-values.

Possible applications to education / intelligent tutoring systems

After contacting Sumit Gulwani, Lee is looking for connections with automated test generation. Tom has suggested jumping off from the bug-fixing work and applying to this problem. Since there are a lot of projects already scheduled, Lee is looking for opportunities to expand or connect our current projects to this work.

Project updates and planning for tangible results

Zeke: Working ont he logical elements of his chemistry system, looking to use spectra rather than graphical representations. Having issues with the ordering on the molecules (“Atom 0 problem”). Omri’s suggestion is to use the heaviest atom as the starting point. The plan for August is for Zeke to hammer out the representation and start evolving molecules as soon as possible.

Omri: Working on a new Clojush interpreter with scoping options, including a scoping specification called “periscope.” No recent work on the flea market. Goal by the end of August is to have the flea market environment available as a experimental evolutionary system. Still having issues with pathological cases for essential vs. nonessential goods.

Kwaku: Currently working on modularity isses.. Taking the top 20 individuals and their modules and re-inserting these seccessful program’s modules in the population. Currently looking for good problems.

Tom: Doing work on lexicase selection and exploring extant greedy approaches and current work on production systems. Lee believes that a graph or lattice-like strucutre that’s pre-computed will make this computation fast. Also working on kata bowling; have found solutions for easy configurations; none for hard configurations. Some solutions have been found using lexicase.

Emma: Possible future meeting topic: tutorial on how to use the database.

Next week’s readings:

Spreadsheet Data Manipulation Using Examples (sent in email), Sumit Gulwani, William R. Harris, Rishabh Singh

Example-Driven Program Synthesis for End-User Programming (sent in email), Marin C. Rinard

Dimensions in Program Synthesis, Sumit Gulwani

Also, for the theoetical background consider this paper on version algebras.

Plan:

Admin

New group readings?

GEVA-clj

TSM

Automatic programming prior work and direction; Microsoft work and generative testing (from Programming Clojure)

Notes:

**Administrative**

Lee’s not yet “dug out” from being away…

we’re meeting next thursday @ the regular time.

WE WILL BE MEEINT @ EMMA’S PLACE IN NORTHAMPTON 89 MARKET ST

finalizations will come electronically

talked about some funding stuff and potential quantum computing grant

**Regular meeting**

we’re talking about p values, statistics & COSMOS – emma

we should pick a reading for our next meeting or meetings… -interested in microsoft automatic programming -emmas stats stuff -generative testing from within book chapter

THIS NEEDS DISCUSSION FROM LIST

–Talked about grammar based stuff from irland geva – cla

–tag space machine –emma’s going to build it.

Admin

• GECCO receipts – get to Lee with signed forms

GECCO thoughts

• Our talks were well-received
• Maarten Keijzer ideas: Push Fourth (one stack), Bad Hash Function (tag based on content)
• GP best paper may be related to our work
• Discussion of GP as optimization vs. program induction
• Discussion of automatic software repair and Simon Harding’s multi-type Cartesian genetic programming.
• Emma: could see GP as a front-facing part of other hybrid ML algorithms, or as an exploratory technique to understand data better, as opposed to a last-ditch effort when other things don’t work.

Steady-state GP

• Lee gave a description
• We may be interested in trying out steady-state PushGP
• Also, not sure of relation to ALPS (Age-layered population structure)

Rewarding Tag Usage

• Kwaku and Emma plan to investigate rewarding tags
• Encouraging modularity explicitly in the fitness function may be useful
• Difficult to know what to reward
• Other tag ideas include a shared tag-space in the whole population, or hereditary tag spaces

Blog

• Emma wants to start a blog to encourage communication with the rest of the EC community about issues we think are important
• May concentrate on understanding problems

Admistritavia

• Matlab: We now have access to 3 Matlab licesnses and various modules. The Simulink module only has 2 licenses available and is to be used for Lee’s collaboration with Prof Kourosh Danai. If you are interested in having access to Matlab, email Lee directly for instructions on the install. IMPORTANT: when using Matlab, you will need to explicity release your license (i.e. do not just put your computer to sleep).
• Josiah would like to see someone become an expert in R.
• Next lab meeting: Friday, July 13th at 1:30 in our usual room.

Discuss paper : Benchmarks

Main issues: privledging of MBF and ignoring outliers, few programming problems suggested, no dynamic problems (Lee and Tom)

Suggestion for a non-trivial programming task: evolving CPS expressions for automatic compiler generation (not terribly compelling, but an example of a programming task that’s hard for humans but easy for machines)

This paper is discussed in the GP-1 session on Monday.

GECCO

(*) Lee would like someone to consider attending Stephanie Forrest’s Evolutionary Software Repair Tutorial that occurs during the second half (i.e. after Lee and Emma have presented) of UP workshop

(**) Suggestions for this time frame: Evolutionary Computation Software Systems Workshop (two blocks), Generative and Developmental Systems (latter block).

(***) Consider attending the IGEC/SS/SBSE Best Papers session or the GBML 4: Reinforcement Learning

(+) Consider attending GA 2: Dynamic Optimization & Exploration Techniques

(++) Consider attending Alife 2 Best Papers or GP 4 Best Papers

(+++) Consider attending GP 5: Empirical studies, which features Krzysztof Krawiec’s paper.

The poster session will be Tuesday evening.

Everyone should atted the SIGEVO meeting Wednesday morning (and also the two keynotes on previous days). Individual tracks will have their own keynote speakers. Lee will be attending Stuart Kaufman’s talk.

Everyone should at least read the first sentence of all abstracts from the proceedings when they get them!

Consider attending the best paper sessions.

Bill Tozier will be around throughout; Lee and Tom to chat with him.

SIGEVO meeting Monday evening for Lee.

Goals: network and don’t be shy about introducing yourself to someone.

• Omri gave his status update
• Tom refactored Clojush and everyone was happy
• Decision: After GECCO, we will meet on Friday (not Thursday), July 13th at 1:30PM
• We discussed whether we might discuss papers related to the Humies
• Decision: Omri will select a few candidates for a paper to read from the GECCO papers (this reading will be for the upcoming meeting on July 5th)
• We talked about Clojush ECJ integration
• Decision: Every member should set and make known their goals for the rest of the summer
• We talked about the Nimrod problem. Lee presented interesting “binary list recursive concatenation idea” (I just made that title up).
• We talked about making tags floats in Clojush, but decided it was not necessary at this time

Administrativia (including agenda/notes management)

Discussion of Krawiec’s GECCO 2012 paper

Discussion of items raised in status reports, possibly including:

• scheduling large numbers of runs on fly
• improving the speed of lexicase selection
• description of the kata bowling problem and its significance
• new formulation of the Nimrod problem
• rational tag machines (see https://hampedia.org/w/images/c/c2/Rtm.pdf)
• Omri’s associative map idea?

NOTES FROM MEETING:

we talked about the paper.

group todo item: talk to krzysztof at conference

minimal tag machine (Lee)

•  evolve a tag space
•  pairs & lists
•  people welcome to work on this on its own thing or as part of push.

Bowling problem

• introduction to problem
• Tom is still working on it
• maybe problem is a translation problem… z means 1, etc
• maps

talked about lex case fitness selection
-talked a lot about how long different ways would take to run

next week:
-nimrod

1:30-1:40 — Administrative Details

This is where matters that can be concluded with a quick conversation should be taken care of. If there are any quick one-on-one discussions that can be done during this time, they should happen here. Students not engaged directly in these conversations can be helping Omri get set up for his practice talk.

1:40-2:00 — Omri Practice Talk

2:00-2:15 — Feedback on Omri’s talk

2:15-2:45 — Paper discussion – Altenberg (For next week, read Krzysztof (Chris) Krawiec’s paper)

2:45-3:30 — Misc

• Discuss collaborative work space
• Discuss Clojush refactorization

In attendence: Omri Berenstein, Thomas Helmuth, Zeke Nierenberg, Lee Spector, Emma Tosch

Administrative details and getting started with Clojure with Zeke

Emma’s talk

Lots of talk about Emma’s talk

Progress reports

• Emma : talk prep, working on web interface for launching and analysing runs
• Tom :
• Omri : talk prep, working on scoped clojush
• Zeke : getting settled into using Clojure
• Lee :

In attendence :  Lee, Kwaku, Tom, Zeke, Omri, Emma

kwaku’s talk
talking too fast
dynamic stuff – need to be a it clearer – did he address why bloat is a problem?
too much stuff on the results page
space out slides
practice talking without the slides – maybe write out speech and then add slides into the talk where appropriate, then ditch the talk
if there are words between slides in

the talk, make them into two seperate slides
focus on why the problem is interesting

tom status
email back from jensen on evolving classifiers – community not that interested, due to community interest in accuracy, rather than interpretability
– what about pitching marginal cases? / outliers
hod lipson – eurika – scientific theories based on data – ML guys think the overfitting part is uninteresting
modelling individual differences using LDA – connection to EFK

zeke
cycloformation? – formation of new carbon ring something something
rings = important
rings = dangerous to make, trying to make reactions happen in water, rather than something dangerous

In attendence : Lee, Kyle, Tom, Emma, Kwaku, Omri

three fundamental features of biological evolution (from abstract)
1) particulate genes carry some subtle consequences for biological evolution that have not yet translated mainstream EC
2) the adaptive properties of the genetic code illustrate how both communities can contribute to a common understanding of appropriate evolutionary abstractions
3) EC exploration of representational language and its role in the genotype/phenotype debate

Regarding #3 — consider some of the points in the Language Instinct. We don’t want language to change, even though the meaning change – better said, we don’t want a faithful transcription of sound via characters because these aren’t robust to change. think about the mit article I forwarded – these loop thingies aren’t calculating exact solutions, but approximate solutions. Relationship between GP/GA and other stochastic learning systems (and probabilistic systems) and approximation algorithms.

can approximation algorithms make seemingly inherently sequential problems into parallel problems?

“conscious intelligence vs natural selection”
i.e. engineering vs science
i.e. optimal solutions vs approximate solutions

randomness not just of the RV but of the model?
meta-randomness
second-order randomness
probabilistic programming languages as second order randomness?
need padding in randomness, redundancy to ensure robustness

buidling a system that is sound and consistent but not complete – but GPs are already turing complete

need probabilistic analysis in the test cases – starting with concise representations of the test cases’ domains – think like an ML – domains are sets that can be represented in closed form or with a generating function

what is the purpose of having EC mimic biology?
(1) to make adaptive programs, following the intuition that evolution is a powerful force we need to come to understand
(2) to better understand biology and complex stochastic systems

this paper references Ostrowski and Reynolds as people who are studying EC from a search perspective

(1) PARTICULATE GENES AND POPULATION GENETICS
(2) THE ADAPTIVE GENETIC CODE
tom’s thoughts:

(3) THE DICHOTOMY OF GENOTYPE AND PHENOTYPE

genetic drift as a desirable characteristic

authors posit that particulate genes will help redefine recombination. i am still curious about the role of redundancy in EC and whether it has been investigated – connections to language, to probabilistic language classes

my angle: redundancy – it’s everywhere! doesn’t conflict with the bayesian stuff either

how to balance small populations, where mutations are more pronounced, with the dilution that occurs over larger populations over many generations – maintaining locality (and perhaps a linearity of fitness) while viewing a population-wide (ie across subpopulations) concentration of parameters/measure/etc.

IMPORTANT: “Unless offspring are infinite in number, their allele frequencies will not accurately mirror those of the parental generation, but instead will show some sampling error (genetic drift).”

“In effect, pariculate genes in finite populations improve the evolutionary heuristic from a simple hill climbing algorithm to something closer to simulated annealing under a fluctuating temperature.”

“co-adapted gene complexes” – Fisher, 1930 and O’Reilly 1999

Adaptive cookbook = REDUNDANCY

error minimizing code smoothing the fintess landscape? – comes from upublished data

lee’s stuff
tag space machine
works with stacks, all code lives in the tag space
ratio tags, denseness of numbers
cucumber – got the R spec and cucumber book
smaller GP steps into cycle
some red/green thing?
1) penumbra
2) keep cool? – climate change negotiation game between countries – can we find or quickly write a simulator?
3) Zeek Nieremberg – new lab member – filing Div 3, mostly a natural science guy
4) physical space, course partcipation – GP and intro class on creativity

tom stuff
1) size-based nodes for tournie selection – turn into journal article? tom’s not that interested, but it is low-hanging fruit; IEEE transactions on EC – publish more things like case-studies
2) evolving classifiers research – emailed Jensen
3) something more substantial with tags – Lee has faith in inexact matching – can we find a way to fit this into probabilistic matching? well, matching is probabilistic anyway via the gp mechanism; maybe the real thing
sean luke – GECCO paper – benchmarks in GP

emma stuff
get something for the evolutionary computation journal
more theoretical people are in that journal
extend to other EC systems
extend beyond EC systems to other stochastic systems

kyle stuff
dissertation – coevolution, lots of pages
kyles talk – autoconstruction
can order be reduced to ILP? We know that ILP is intractable.
what about picking a problem that’s easier with probabilistic guarantees?
what about running TSP vs clique
how do you measure “useful genetic material?”
run cosmos on the problems – look at the behavior