Paper presented at the Third Annual National Meeting of the American Synesthesia Association in New York, Rockefeller University, 2-4 May 2003.
This page contains the text, to see the slides of the presentation click here.
Crétien van Campen & Clara Froger
Yesterday and today, I have heard fine talks by artists and researchers. I appreciate the way the organization has succeeded in organizing a meeting of artists and scientists from various disciplines. In this presentation I want to tell you about a Dutch collaboration of an artist and a researcher in the study of synesthesia.
The intention was to do a duo presentation by Clara Froger and me. Unfortunately Clara is hindered to come to NY. She apologizes and sends her best wishes to you all. At the exhibit of synesthetic art you can examine some of her works.
I am working as a health care researcher in the Netherlands. Besides my work I have studied synesthesia in science and art. Clara is an artist and color consultant. We developed the Netherlands Color Synesthesia Method that was presented by Carol Steen at the ASA conference in San Diego last year. An article will appear in the coming August issue of the American journal “Leonardo”.
Our backgrounds stretch beyond our personal lives and go back into European artistic and scientific traditions. As you all known the study of synesthesia has a long history in Europe. This figure gives an impression of the interest in synesthesia by artists and scientists in the last centuries.
Synesthesia is not the only perceptual phenomenon that has been studied by artists and scientists. It looks that artists before have explored almost every scientific subject of perception. Just to give you an impression of some phenomena that were studied by artists and scientists: the perception of linear perspective (Dürer), the perception of simultaneous contrast in color (Seurat), the perception of reversible patterns (Rubin), and the perception of symmetry and mathematical puzzles (Escher).
In most cases artists discovered a phenomenon in their practice of sketching and painting. They explored the phenomenon and applied it in their art as they saw it. They used a first person perspective. Often, scientists rediscovered the phenomenon later, tried to isolate it under laboratory conditions and explained its underlying physiological and mental mechanisms. They used a third person perspective to establish objective mechanisms.
The elucidating article on first and third person perspectives in synesthesia research by Daniel Smilek and Mike Dixon provides a fruitful conceptual framework for our study. We consider our experiment as an illustration and elaboration of this distinction.
[Nick double bass]
We combined first and third person perspectives in a special way. One month ago we invited nine synesthetes to the BMBR gallery (Beauty Must Be Realized). All nine have colored-word synesthesia (most of them of the colored-grapheme type) and four persons hear music in color too.
We asked them to do two things:
[color chips]
1. They chose color chips that
represented best their synesthetic color responses to specified stimuli.
[participants paint]
2. They painted their synesthetic
responses to a selected number of the same stimuli.
[Cretien reads words]
The selected number of stimuli that were
painted consisted of five spoken sounds: O E OE KOEL STROEP.
[Nick and Clara]
And six musical sounds: the D-chord
played in major and minor key successively on double bass, guitar and trumpet.
The selected color chips were used to analyze the consistency of the persons color-word and color-music relationships. Consistency tests are now widely used in scientific study of synesthesia. In addition to the traditional consistency test, the Netherlands Color Synesthesia method computes consistency scores for each of three dimensions of color synesthesia: i.e. hue, chroma and lightness.
So, we gathered two types of information for each participant: a first and a third person perspective of his or her colored-word and colored-music synesthesia. At the moment we are analyzing the results that we gathered only one month ago. We would like to present some of the first results and share our first interpretations with you. We like to hear your suggestions.
[Colors of the Dutch word KOEL (=
cool)]
This slide shows two responses to the
same stimulus: the spoken word KOEL, meaning cool in English (as you see some
Dutch are bilingual; this person painted two interpretations of the sound)
On the left side you see the color chip
that was selected for the word KOEL by this person in the test. An on the right
side, you see a visual representation of the same word by the same person.
Comparison:
1. the painting shows more nuance (see
e.g. the change of blackness and chroma in the O of the Dutch word KOEL)
2. the painting shows not only the color
but also the form and space of the color (see the clouds around COOL)
On the other hand the color chip has
direct computable information on the dimensions of the color. The code for this color chip [6030 R70B]
indicates that it contains: hue = 70% blue 30% red; chroma= 30% of the pure color; lightness= 60% black 40% white.
Later, the person commented that the
real color is more transparent than the selected color chip. The paintings
capture the transparency and space of the color better than the color chip.
[Influence of context on colored
letters I]
This slide shows that the method of
painting is better suited to show the effects of surrounding letters on the
color of a letter. This subject made five paintings of the spoken sounds. In
Dutch the combination of O and E produces a different sound: OE. This person has
apparently colored-grapheme synesthesia.
In the test she chose a light ochre yellow and a muddy yellow for the word KOEL. The painting shows that the O is black, her E is bright ochre yellow. When she hears the different sound of OE she sees the colors of O and E. But mentally, they mingle into a muddy green-yellow. The words KOEL and STROEP below show that more letters influence the colors of the OE.
[Less influence of context on colors]
The next slide shows how individual
synesthetes can differ. For this person the color of OE is green (on a yellow
background and a shade of white on top). And it stays green irrespective of the
surrounding letters, as you can see in the words KOEL and STROEP.
The green OE is made of the white O and a green-yellow E. Remarkable is that the colors mingle in the words KOEL and STROEP but not in the isolated sound OE.
Note: Most participants mixed the colors of O en E to represent the color of OE.
[Sounds of musical instruments]
In the musical session, a D major chord
and a D minor chord were played on double bass, guitar and trumpet. This person
has selected in this test the same color chips for major and minor chords; but
for each instrument another color. The paintings show a nuance that the test did
not show; namely the difference in sound between major and minor chord. As the
sounds are higher they become lighter and more transparent, according to this
synesthete. Note 1: Most participants used
“earthly colors” for the double bass.
Note 2: Minor chords are painted
redder/warmer than major chords by most participants.
[Comparing two types of information on
colored-music synesthesia]
We can compare the painting of this
person with his color synesthesia profile that is produced by the Netherlands
Color Synesthesia method. The profile (at the right side) reveals
a remarkable aspect of his color-music synesthesia. If one considers his
synesthesia in three dimensions, it appears that it is built on two strong
pillars and has one weak pillar. He has a strong sense for the lightness and the
chroma of music, but a weaker sense for the hue of the music.
Looking back at the painting on the
left, it is visible that he varies in lightness and chroma but not so much in
hues. His hues seem to stem from the colors of the instruments: e.g. a brown
double bass and a yellow trumpet.
[Improvisation]
An visual musical improvisation as an
intermezzo . Halfway a day of hard and precise work we paused with a free
improvisation on double bass by our musician Nick McGuire. All participants were
allowed to paint whatever they wanted. Here is the painting I liked most.
Strikingly, she is not a color-music synesthete.
[Discussion of results]
In this early stage of analyzing the
results, we already see two differences between scientific testing and painting. The use of color chips restricts the
study of synesthesia to color, while the use of painting shows the aspects of
form, space of color too. (In fact one should be able to represent rhythm and
movement too.) This implies, that using color chips
means loss of information. Painting is a better approach to represent the real
synesthetic experience. But paintings are more difficult to compare. Color chips
have the advantage to be computable and give the opportunity to statistical
analysis of the dimensions of color.
We will continue our analyses the coming
months and hope to profit from your suggestions.
[Conclusion]
In conclusion, we want to make some
statements on the collaboration of artists and scientific researchers.
1. Art and science seem to contribute in
different ways to our knowledge of synesthesia. But we have seen in our
historical review of synesthesia research and in a broader historical review of
visual research that science and art have always been interacting.
2. Artists and researchers can both
profit from critical interaction. The collaboration of Clara and me is an
example. After many years of collaboration, I understand better now what a color
experience in fact is. And Clara has seen the advantages of measurement and
comparing individual synesthetes. We have profited both ways from our
expertise.
3. To return to the conceptual framework
of first and third person perspectives. We recommend artists and researchers to
change perspectives: between first and third person perspective. Not once but
regular. Change from assessment and objective knowledge to personal records and
back and so forth. Finally, we think that progress in synesthesia research is
probably not a linear, but a cyclic process. Thank you for your attention.