GO Nominated Artist: Jonathan Blum
GO Nominated Artist: Naomi Safran-Hon
GO Nominated Artist: Eric Pesso
GO Nominated Artist: Oliver Jeffers
Does engagement require competition?
Now that we’ve published a lot of our findings about the data we saw over the open studio weekend, we are releasing all of the feedback we received when we asked participants to “Share Your Story.” All of the feedback is now available in the Shared Stories area of the GO website; we’ve tagged stories consistently, so you can easily discover what themes emerged. You can also search by participant type (artist, voter, volunteer) and neighborhood, so it’s possible to see the feedback coming from participants in one area and compare it to experiences in another.
As we reviewed the feedback, Sharon and I were struck by how much participants talked about “discovery”—one of the largest tags in the cloud. This feedback has been overwhelmingly positive and inspiring. If you take a look, you’ll find participants discovering so much about their communities and neighborhoods and expressing their gratitude for the project.
Voters canvassed their neighborhoods finding familiar faces in places that were not accessible prior to GO; artists reported a greater awareness of others working near them, often in the same building or just down the street. Many participants felt GO facilitated a greater understanding and a coming together that wasn’t present before the project.
In our previous posts, we discussed much of the feedback we’ve seen. The data suggests that while hot spots existed, the traffic during the open studio weekend didn’t necessarily align with people’s assumptions. We discussed the weekend’s hours and participants desire for a multiple weekend event. Plus, we received plenty of feedback about technical problems, our app, our website, and even our printed maps; we’ve talked about this quite a bit. Despite all these hurdles, you’ll find that almost every story says something along the lines of “We must do this again.”
The common thread you’ll see running through all of the feedback is that this project made a difference to people, grew communities, and fostered connections that participants had not anticipated—people were deeply engaged through GO. Yet, as much as we heard rave reviews about the discoveries people made over the weekend, we also heard people struggling around the notion of this project as a “competition.” This incredibly rich and interesting feedback about the “competition” combined with participants’ overwhelmingly positive experience over the weekend has led us to wonder, if GO had been an open studio event without an exhibition, would your level of investment—or that of others—have been the same?
Learning Lessons Going Mobile
This is the final post in a three-part series about the use of mobile technology during the GO.
During the open studio weekend we wanted to provide as many mobile options as possible, but simplification could have helped us a great deal. We spread ourselves too thin in a quest for accessibility.
Despite the comment below and similar sentiments shared via other forms of feedback, Sharon and I still believe the high bar is necessary, and we wouldn’t change the requirement of registration and seeing many studios in order to nominate; we also believe that removing the nomination process from the weekend activity allows for needed reflection. However, within this set of requirements, we can and should simplify the means to get there.
As I look at these posts about texting and iPhone use, there are many lessons to be learned. If I had to do this year all over again, we would ditch texting in favor of telling people to write down codes, and we would have eliminated mobile web in favor of more staff time allocated to the iPhone app. Though we ran into unforeseen problems, I think having more time could have only benefited us and the additional time would have meant having more hands on deck when we started to see issues crop up because we would have been supporting fewer options.
As a powerful visualization, take a look at a detail of door signs which were on every artist’s door; this is the area of the sign that tells people how to check in to a studio and I’ve added arrows for emphasis:
We worked countless hours on this signage to try and make all the options as clear as possible, but you can see, there’s just too many choices and this results in too much information that’s not as clear as it could have been. It would have been a lot simpler if we just offered two methods—use our app or write it down. Not only would this have helped focus our tech efforts, it would have helped us better communicate about the choices available and we wouldn’t have been spread too thin in trying to support too many options.
Sometimes, we need to remember the wisdom of Mies: Less is More.
Growing Pains with an iPhone App
This is the second in a series of three posts about the use of mobile technology during the GO open studio weekend. The first looked at basic trends and SMS texting; this post will delve into the iPhone app, and the last will consider general lessons learned.
Few people know that we almost pulled the iPhone app from the store right before open studio weekend. Late Friday afternoon, the app was rendered totally unusable as it crashed at every tap and we wondered if it would be a better user experience to pull the app in favor of using other less crash-prone methods to check in.
As it turns out, for as much testing as we did, the one thing we couldn’t replicate easily was the load on our servers going into the open studio weekend and the app wasn’t handling spikes in server traffic very well. After a very late night we managed to stabilize the situation prior to the open studio weekend, but not before we frustrated a lot of participants.
I wish that was the extent of the problems we faced, but there was more. At first load, the app would check to make sure its list of studios was up-to-date. In the week leading up to the open studio weekend we had about a hundred artists cancel their participation, so the app had a lot of work to do to update. You can imagine what’s next…yup…crash. Once the app crashed on the initial startup, it wouldn’t crash again, but this was on the heels of the Friday night issue and there was plenty of frustration with no easy way to tell users to hang in there.
We went into the weekend knowing there were problems, but that most had been resolved or would be resolved in good time. We had no idea if participants would stick with us or ditch the app in favor of the other platforms we offered, so we held our breath and waited to fight more fires, but I remember seeing this tweet at some point over the weekend and feeling some sense of relief that someone out there was having a good experience.
There were some additional positive experiences around the app that came as part of the “share your story” feedback, but it felt as if these were the exception to the rule. So, I was floored to see this big, blue sea of check-in data in the chart below—despite all the trouble, this app was used quite a bit. Given this popularity, we are not sure if the app worked mostly as intended or if people used it with continued trouble. Were the people having trouble just louder about it or was their experience indicative of most users?
That said, one more glitch cropped up during the weekend. After a while, the app stopped displaying users’ most recent check-ins. Even though the check-in would register on our side, the app wouldn’t tell the user. Some folks told us they went home to find all their check-ins and figured out this was a display bug, but we know we probably lost others in what I’d deem a user experience problem of the highest order.
In terms of functionality, we heard mostly good feedback, but not all, about features we provided. Participants reported that the mapping feature was solid and being able to follow an itinerary a real win, but the data we saw around the app’s nomination functionality found this feature to be of little use; most people turned to web when casting nominations (web 84%, mobile-web 10%, mobile 5.1%). The nomination functionality was a late day addition because I was absolutely (and incorrectly!) certain that users who started with one platform would expect every step of the process to be included. As it turned out, most users only used mobile when it would be most efficient (open studio weekend), and they turned to web once the need to be mobile had elapsed. Note to self: avoid the feature creep—it’s always better to start with a set of features and then wait and see what people ask for before attempting to develop for every situation.
Tomorrow we’ll talk through all the lessons learned, but in the meantime we are curious about your own experiences with the app. Did your own experience match up with the trends we were seeing with the data and the feedback?
Mobile Use and Texting Pains
This is the first in a three-part series about the use of mobile technology during the GO open studio weekend. Today, we’re going to look at basic trends and talk in depth about the use of texting; later this week we’ll look specifically at the iPhone app and lessons learned.
When we first looked at platform use during the open studio weekend, we were a little surprised at the data; when voters checked in to studios, they used the website almost half the time.
As we started to look at the data over time, we started to understand the trends a bit more. You can see from the chart below, the iphone app dominated during the open studio hours and web traffic started to increase substantially at the end of each weekend day and on Monday, during the check-in grace period. You’ll notice at the exact times we saw a drop off in traffic to studios (6pm), there was a rise in web traffic indicating people headed home to input the artist codes they had collected throughout the day.
You’ll also notice a big dip in traffic on Sunday around 7-8pm; that’s right around the time the website crashed, so it’s a good thing we had established the following Monday a grace period allowing those writing down codes and anyone who experienced tech difficulties enough time to check-in—as you can see, people really needed it.
When we look at mobile web, the numbers there are pretty much as we’d expect. iPhone users account for 81% of the mobile traffic on our website (true for both gobrooklynart.org and brooklynmuseum.org), so when it became apparent we only had resources to develop one app, the iPhone had to be the platform of choice given what we knew about our audience’s device carrying habits. While we didn’t have the resources to provide Droid users with their own platform-specific app, we were able to create a slimmed down mobile website which could be used for checking in to studios and following an itinerary; a 6% usage rate doesn’t seem too out of line with our audience’s overall mobile usage.
Texting usage was another matter entirely. We were stunned at the low use; in our minds, texting was the most accessible check-in method. As it turns out, the statistics seem to indicate the average participant found writing down numbers and entering them on the website (43.7%) way more accessible than texting (9.2%).
For as simple as we thought texting would be, there were many problems with it. Participants reported not understanding how to text to a alpha shortcode and this was especially true for users who had QWERTY keyboards with no corresponding numeric values. So, for instance, if we said “text the artist code to BKLYN” many participants had trouble understanding this meant “text the artist code to 25596.” Other users got hung up when it came time to sync their phones with their web accounts—a needed step in order to match a user’s phone with their online registration.
What interests me most about all this is when conceptualizing GO, we wanted to ensure there would be an “accessible technology” component; something everyone had access to because we knew that so many people elect not to have a smartphone. Even though participants had universal access to texting, the accessibility became questionable in its complications. In hindsight, the most accessible mobile method had little to do with technology at all—pen and paper proved the method of choice and people seemed more comfortable coming back to their computers to complete their weekend participation.
In the next post, we’re going to delve into the iPhone app and later we’ll talk about overall lessons learned and things we’d do differently next time around.
Only So Many Hours in a Weekend
Of all the feedback we’ve received, issues around GO’s hours really bubbled to the top. A common refrain from voters and artists alike was one weekend was not enough; so many people have suggested we split the event up over multiple weekends. Sharon and I are sympathetic to these concerns; each of us created overwhelmingly large itineraries and spent the weekend seeing as many studios as possible throughout the borough, but for as many studios as we managed to visit, we couldn’t see them all. However, of all the changes we may consider with possible future iterations of the project, the idea of splitting up GO over multiple weekends isn’t one of them.
When planning GO, we discussed the idea of a multi-weekend event at length and, in the end, decided against it. A multi-weekend event would mean breaking Brooklyn into zones because neighborhood boundaries are too fuzzy to be helpful in this scenario; zoning seemed antithetical to the project. If we did split things up, we worried about issues outside of our control creating greater disparity and unfairness. The bad weather we experienced on Saturday or the issues with MTA service are both great examples of this. It’s one thing if you have bad weather and we all experience it together, but it’s a totally different story if the bad weather falls on weekend your neighborhood has open studios when another neighborhood benefits from perfect conditions. Of course, we could run the project over multiple weekends throughout the entire Borough, but we didn’t think it would be realistic to ask artists to be in their studios for more than two days.
Speaking of artists, we heard time and time again that the open studio weekend hours—11am to 7pm for two consecutive days—was just too much. As we look at what worked and what didn’t, we wanted to dive into the check-in data to see if GO’s hours worked for both artists and voters. David Huerta, one of our web developers, pulled together the data so we could take a look.
As it turns out, there’s a massive drop off at 6pm on both days and it’s pretty clear the day could have ended a bit earlier than we had scheduled it. We also wonder if noon might have made a better start time, but there’s a pretty clear jump start to the day and for families with children, starting early can be key in their participation.
Above it all, one of our aims was the hope we could bring Brooklyn together through the project and we believe having the event occur on one weekend across the entire 73 square miles of the borough—1700 artists opening doors at the same time—was something special and powerful. In hindsight, we might have changed the hours of the open studio weekend, but we wouldn’t have changed the weekend for the world.
The Sweet Spot
With the artist nominations just announced, we wanted this next post to take a critical look at the nomination data and consider the results. Remember this heatmap of check-ins over the weekend?
At first glance, most people looked at this map and assumed the majority of nominees would be coming out of the hottest of hot spots, but the resulting nominations are surprising because they run counter to these assumptions. Compare to the heatmap of nominations:
You can see what’s happened here—nominations were dispersed throughout. Even though an artist may have been in a high density building or even a hotspot, the nominations in those areas were dispersed among the many studios open. Was there something in the traffic patterns and the resulting metrics that meant a confluence of events could occur—was there a sweet spot?
Corinne Hoener, one of our web developers, is going to show us a couple of examples:
There were 1708 artists in 758 buildings with open doors for GO. Of those buildings, 188 housed multiple studios. If we look at the data on our top ten artists, six of the ten were in multiple-studio buildings, and five of those six were in buildings with 8-15 studios, such as 925 Bergen and Invisible Dog (both had 15 studios). Naturally, this led us to wonder if 8-15 studios is a “sweet spot” in terms of having enough studios to draw visitors to your building, but not so many that visitors are overwhelmed by choice. If we crunch the numbers, it certainly looks that way.
On average, an artist’s studio saw 30.35 check-ins, but artists in buildings with 8-15 studios had higher than the average with 44.09 check-ins. On either side of this sweet spot, there were fewer check-ins; artists in buildings with fewer than 8 studios had 22.92 check-ins and buildings with more than 15 saw 40.94 check-ins. Studios in buildings with 8-15 studios also received about two more nominations than average. That might not sound like a lot, but with tight margins like we have, two additional nominations can make all the difference.
Speaking of nominations, it’s important to remember that an individual visitor can check-in to as many studios as they want, say 100 or more, but they can still only nominate up to three artists. So if we’re digging into the numbers behind our top ten artists, we have to consider the number of visitors to their studios. And again, we can see a sweet spot effect for those buildings with 8-15 studios. Among our top ten artists, studios in buildings with 8-15 studios saw on average 223.25 unique visitors. These buildings had, on average, 12 studios in them, so there is an average visitor-to-studio ratio of 18.6. Contrast this with two mega studio buildings, which had an average of 72 studios in them and an average of 429.5 visitors—nearly double the average visitors that the studios in 8-15 buildings saw. However, since they also had about 6 times the number of studios, that leaves them with an average visitor-to-studio ratio of only 6.0. There were far fewer visitors relative to the number of studios in these major buildings. And as we pointed out earlier, fewer visitors means fewer potential nominations.
There’s also some interesting data related to intra-neighborhood traffic, the visitors going to see artists in their own neighborhoods. Across all the neighborhoods, an average of 32.69% of check-ins came from visitors who lived in or near that neighborhood. However, across the nine neighborhoods represented in our top ten, on average 42.07% of check-ins came from people who live in or near those neighborhoods. In fact, seven of those nine neighborhoods had intra-neighborhood percentages that were above average. It’s possible that this factor combined with a sweet spot (or even the high intra-neighborhood traffic alone) can help to boost nominations in a rally effect. Intra-neighborhood stats have now been added to the raw data, so you can take a look.
We should recognize that the artists who are in the top ten are there for a reason. Sweet spot or no, many artists were nearby (some even in the very same building), but it’s the work of these ten that resonated. Also, we didn’t see either sweet spot traffic or the rally effect cause a “sweep,” or dominance of one area or building in the nominations. Even so, as Sharon and I think about possible future iterations of the project, we wonder how can we avoid the sweet spot and continue to level the playing field.
Simply put, in order to lessen the effect of sweet spots, we need a critical mass. With 1,708 artists, we think we’d need at least 17,000 eligible voters going to at least five studios each. Though we estimate we had 18,000 voters, only 4,929 were eligible to nominate artists; many of our participants were just visiting studios without checking in or nominating. We both feel that the high bar to become an eligible voter is critical and non-negotiable in future; we would also want to continue encouraging all kinds of participation (voting and non-voting). Given what we saw this year, we think the metrics to get us to statistical significance and wisdom of crowds decision-making would be a tough uphill climb at any point.
As Sharon looked at the results, she found interesting work not only at the top, but also among the middle of our long tail (and beyond), so as we think about future iterations of the project, we find ourselves asking about participation models that could surface even more work for consideration.
The good news is, this first go round has given us an incredibly rich place from which to start. As we consider the future of GO, we can experiment with various ways to move forward using this year as a solid foundation.
Your Ten Nominated Artists
After approximately 147,000 studio visits to 1,708 artists, and then 9,457 nominations, we have our top ten nominated artists.
In alphabetical order:
- Aleksander Betko, Cobble Hill, painting and drawing
- Jonathan Blum, Park Slope, painting and printmaking
- Adrian Coleman, Fort Greene, painting
- Oliver Jeffers, Boerum Hill, painting, illustration, and drawing
- Kerry Law, Greenpoint, painting
- Prune Nourry, Boerum Hill, photography, video/film/sound, and sculpture
- Eric Pesso, Ditmas Park, sculpture
- Naomi Safran-Hon, Prospect Heights, painting
- Gabrielle Watson, Crown Heights, painting
- Yeon Ji Yoo, Red Hook, mixed media sculpture
We are pleased to have such a mix of artists represented in this group, including painters, illustrators, sculptors, and installation artists. Painting clearly ruled with seven of the ten artists being self-identified painters. At the same time, we note the absence of design, fashion, and textile arts, and also that photography, video, and performance are represented only in Nourry’s work.
The results were also a bit surprising in terms of the weekend activity, as was hinted at in Shelley’s post on unexpected traffic patterns. Nine neighborhoods are represented, but they are not the neighborhoods that most people were predicting to be the hot spots. Shelley will be delving into these results to show how visitation may have shaped nominations, so stay tuned as we report on this more.
As we have discussed along the way, this project aimed not only to bring artists and their communities closer together, but also to open up the curatorial process. While most of our exhibitions at the Brooklyn Museum have the standard model of being curator conceived and organized, we recognize the value in considering other models. Most recently, our Raw/Cooked series features under-the-radar Brooklyn artists nominated by an artist advisory committee and then selected by Eugenie Tsai, our Curator of Contemporary Art.
With GO, we established a collaborative process where we invited the public to visit studios and then nominate artists before Museum curators would visit the top artists and make the determination of those to be featured in the exhibition, and the work to be included. This will undoubtedly be a difficult task, particularly given the great range of work and the strong support for these artists. Nonetheless, we are committed to creating the best exhibition possible within these parameters, and that will mean making some tough choices.
Over the next month, we will visit these ten artists’ studios and begin highlighting them one by one on our website. By November 15th we will announce the featured artists for the exhibition, which will open on December 1, 2012 during our Target First Saturday evening.
Unexpected Traffic Patterns
We expect to announce the nominated artists tomorrow; we’re running just a little bit behind schedule. In advance of announcement, we wanted to share some of the neighborhood data and think about what happened over the open studio weekend in context and in relation to some the feedback we’ve received.
Throughout GO, participants have expressed concern about the process being unfair. We’ve been asked if we’d weight results or adjust data to create an even playing field, but our answer has always been that we’d let the data stand as is; we simply didn’t know enough about traffic patterns or participation from the outset to create a model that would work for everyone. We felt it was best to let the weekend play out and look carefully at the resulting data to see if we would need to make adjustments in possible future iterations of the project. That said, we expected things to surprise us and they did.
Many participants were concerned that studios in low-density areas (i.e., not typical “artist hubs” like Gowanus and Bushwick) would receive less foot traffic and thus be at a serious disadvantage in an event like GO. If you look at the neighborhoods with the most check-ins—Gowanus (6442), Red Hook (5800), Sunset Park (4751), Bushwick (4128), Greenpoint (3959)—at first the findings seem to align with these concerns. However, if we dig just a little more deeply, we find the data reveals far more nuanced results.
Gowanus, Red Hook, and Sunset Park had something in common; each had many studios, most within easy walking distance and some buildings with large concentrations of artists (Brooklyn Art Space, Screwball Spaces, and Chashama, respectively). As a result we saw a higher than average rate of check-ins per studio; basically, a visitor could go into one of these neighborhoods and rack up a bunch of check-ins all at once. However, those same areas were below average in terms of the amount of visitors in relation to number of studios open. In other words, even though there were a lot of studios with high numbers of people checking in, the ratio of visitors to open studios was relatively low in comparison to other areas.
Along the same lines, if we look at Boerum Hill, Prospect Heights, and Ditmas Park, these neighborhoods were not the densest in terms of studios (particularly Ditmas Park), but all had above average visitor-to-studio ratios. Relative to the number of studios, these neighborhoods actually had much higher traffic than the denser neighborhoods of Gowanus, Red Hook, and Sunset Park.
It was interesting to see Bushwick as a bit of an outlier in all of this, especially given the amount of feedback we received assuming this area would have all the visitation. With 241 artists opening their doors, Bushwick had the highest number of open studios of all neighborhoods participating in GO. However, not only did Bushwick receive below average foot traffic (number of visitors relative to the number of studios), but it also received a below average rate of check-ins per studio. This left us wondering about possible reasons; we think several possible factors are at play.
Many artists in Bushwick told us they felt like the requirement for artists in their studios during the weekend hampered visitation. The same people commenting added that those often visiting their studios during open studio events have been fellow artists, and our requirement curtailed typical artist-to-artist visitation patterns. We should acknowledge the incredibly successful Bushwick Open Studios just took place in June and could account for voters going to other neighborhoods where they had not yet had a chance to explore. Our data supports this theory by showing 61.7% of the check-ins logged by voters with Bushwick zip codes were for artist studios outside of the neighborhood, so Bushwick voters were seen casting a wide net in their studio visits. Lastly, we have to remember our data is only based on those who actually checked-in to studios. A common refrain we heard out of Bushwick was participants didn’t feel the need or want to vote, so foot traffic might have been higher, but it was not registered online.
After the nominated artists are announced tomorrow, we will look closer at how these kinds of traffic patterns impacted nomination data, so stay tuned. One thing we do know is almost all of what we’ve seen has run counter to what many had assumed.
Corinne Hoener, our Web Developer, compiled all this data, analyzed it extensively, and co-authored this post. If you’d like to take a deeper look, she’s posted the raw data.
some friends and strangers stopped by and wanted to blow things up
thanks “GO Brooklyn” for the awesome weekend,
You can imagine the conversations that ensued…what happened over there? Was this a good experience or a bad one? Is this sarcastic? We were a tad alarmed with no context, but the next day Hubert sent a video that clued us into what was actually going on…
When he sent us the video, he added:
my curious neighbor showed up sunday night,
first she wanted to call the police because of the “gunshots”,
as she saw us with helmets and earplugs she reconsidered and brought along
her spanish friends …..
How’s that for a fun Friday post?
Moving Toward an Exhibition with 9,457 Nominations
As you saw in Shelley’s previous post, we were thrilled about the level of participation over the open studio weekend. At the same time, we remained anxious about how many of the 4,929 eligible voters (those who checked in to more than 5 studios) would continue their engagement into the nomination phase. We anticipated that we would lose some people, but again, as has often happened with GO, we have been pleasantly surprised with the results.
Our nomination period ended on Tuesday night. We received 9,457 nominations over seven days. Of the eligible voters, 78% of them took the next step to recommend artists. While we allowed visitors to select up to three artists, many chose fewer—23% nominated 1 artist; 10% nominated 2 artists; and 67% nominated 3 artists.
Perhaps the most exciting aspect of this phase, however, has been that many visitors also took the time to leave provide comments and feedback. Using the virtual guest book feature on the website, visitors left a total of 2,289 comments for artists, ranging from short notes such as “great work” to longer, detailed ones about experiences in the artists’ studios. We have also received 361 responses via “Share Your Story.” These have run the gamut from surprises and disappointments to troubles with technology and the MTA, but almost always offering advice if we repeat the project. We will be releasing this feedback in its entirety on the website, but in the meantime, we are highlighting selected stories on the homepage.
So, what happens next?
We will announce the names of the top ten nominated artists on or about September 25. We are looking at the results with a careful eye (and some fancy programming) to ensure that each artist who has been nominated is eligible according to our terms. I will then visit those artists’ studios along with a curatorial colleague and select the work for the exhibition. The featured artists will come from those nominated by the community, but the focus and structure of the show (particularly what themes may emerge from conversations with the artists as well as how many artists will be included) is all up for grabs.
In the meantime, our web team has also been analyzing the statistics from the weekend to better understand the activity. As a bit of a teaser, we have added a heat map to the homepage, which beautifully visualizes the overall activity over the weekend.