Tasting meal results

Sunday’s farm tasting event was great fun. We had a wonderful set of folks who gave us an afternoon of their time, having fun tasting lots of items and just creating a nice atmosphere. We spent a few hours going through the different dishes, then took a farm tour while Joanna compiled the results.

Quick results (full disclosure below) showed that for most things, the on-farm products came out slightly to seriously ahead. In a few cases the non-farm product was preferred. Goat was strongly preferred to venison, and chicken eggs were slightly preferred to goose.
We did our best to keep the tastings neutral and unbiased, but still learned a few things about running an experiment like this. Every item was served labelled A or B, to which we had a key in the kitchen. We rolled dice to determine the order and labelling within samples, such that there was no predictable pattern to whether a store or farm item was A or B in a given item.

We had scoring sheets set up for folks to rate each item 1-5, with 1 being the highest. This taught us a lesson, as many folks (including, shamefully, me) at times reversed things in their heads and ranked things backwards (giving higher numbers to better tastes). We think we sorted out all those mistakes before doing the tallies, but if/when we do this again, we’ll probably have a clearer system in which the ratings are clearly labelled for each item, like a multiple choice test or something. With so much to keep track of, the system really needs to be obvious. Good lesson.

We also did our best to make the preparations identical, but some differences crept in anyway. For example, the store-bought edamame were noticeably larger than ours, so that may have affected the cooking times and thus the quality. But you can’t get everything perfect when you’re serving nine people 16 courses in a few hours from a home kitchen.


Remember, 1 is the best and 5 is the worst. These are the mean scores from our guests only. Joanna and I also tallied our preferences, but we didn’t want our numbers to bias the results (given that we could easily identify which product was which in most cases).

Farm: 2.29
Purchased: 2.43

Several comments indicated that the farm corn was crisper, but the flavor wasn’t much different.

Okra pickles
Farm: 2.43
Purchased: 1.86

This didn’t surprise me; the purchased pickles were far crisper, and ours were a little too vinegary in comparison. The texture may relate to the multiple preservatives/chemicals in the ingredient list, but the effect was still there.

Morningland (MO): 2.57
Farm goat (MO): 1.71
Cabot (VT): 2.00
Irish: 1.86

This was a bit surprising; we like our cheddar but didn’t expect it to beat out Vermont and Irish cheddars.

Boiled chicken eggs
On-farm: 1.71
Organic Valley: 2.71

I thought this difference was fairly subtle, but the results were fairly clear. This was a close comparison, as Organic Valley is widely regarded as the best of the large-scale producer brands, and our eggs are coming from winter chickens who still are eating mostly grain and not a lot of foraged protein. Our yolks were noticeably yellower, but the flavors were only subtly different. If we can beat the best store option at the low end of our chickens’ flavor potential, that’s not bad. And it speaks well for O.V., as I’ve had far blander store eggs before.

Edamame, frozen in-shell
Farm: 1.83
Store: 3.00

The flavor and texture were better with ours. The store version were larger than ours, so perhaps the cooking changed things, but we prepared both according to the bag’s directions to give them the benefit of the doubt. Folks who had bought our fresh edamame agreed that the frozen ones weren’t nearly as good, but they still ranked above the store-bought.

Edamame, frozen shelled
Farm: 2.43
Store: 2.86

The pre-shelled frozen was noticeably less good than the in-shell. Clearly keeping them in the shell protects the flavor somehow. This was effectively a tie, though it’s worth noting that the farm frozen ones we used in both tests were our least favorite of four varieties grown last year, plus they were our seconds from market harvest (those rejected for market sale). So anyone buying them from us and freezing them got a better bean, but these still tied or beat the store’s.

Goose vs. chicken eggs (scrambled)
Goose: 2.14
Chicken: 1.71

This was a surprise, as we generally think the goose eggs taste richer and better. However, in the blind test the difference was more subtle. One person thought the goose suffered from being unusual, such that the familiar taste of chicken made it rate higher on instinct. Another possibility is that the goose eggs get tastier as spring arrives, and their diet becomes fresher and more diverse (right now they’re mostly eating grain and hay). The eggs we sold at market last year in late April and May got rave reviews, and I agree that the current ones are relatively mild. The textures are noticeably different when scrambled, as goose froths up and becomes airier than chicken.

Dried soup beans
Farm mixed heirloom: 1.57
Store organic pintos: 3.14

This one was obvious, and backed up what market customers told us in the fall and through the winter. Our heirloom beans taste way better than standard beans. We have to charge a lot to make them economically practical, but they are definitely a culinary treat.

Fresh-ground farm corn: 2.14
Bob’s Red Mill cornmeal: 1.57

This one REALLY surprised us, partly because the night before as we cooked up the polenta, we found the Bob’s taste to be strongly bitter (rancid) to the point of spitting it out. We were sure it would be blown away in the test. Yet after sitting overnight, and a quick shot under the broiler, they were fairly similar in taste with only a hint of bitterness in the Bob’s. Also, we found the texture of our meal to be far smoother, with the Bob’s being far clumpier, and expected that to count against it. Someone suggested that, ironically, the clumpy texture of Bob’s and the ultra-smooth texture of ours had led people to subtly assume the farm meal was the clumpy one and rate it higher. Who knows. In this case we clearly lost, but after tasting (and smelling) the just-cooked Bob’s cornmeal the night before, I’m so glad we grow our own. I’m not sure what chemical changes the broiling made, but we’ll stick to ours. Given the already rancid early flavor, it’s not comforting that the “Sell by” date on Bob’s is July 2011.

Basic tomato sauce
Home-canned Sunny Acres tomatoes: 1.57
Muir Glen organic canned tomatoes: 1.86

This was closer than we thought it would be. Like the polenta, we made the sauce the night before, and on early tastings the Muir Glen version was nasty, with a strong canned/chemical/salt flavor. But again, after a night of sitting with the other basic ingredients of our garlic and dried basil, the two sauces ended up being pretty close in flavor. Maybe cooking mellows the flavor of some of the preservatives, though several tasters did identify the distinct canned flavor. Once again, like the polenta, even where these are a draw we’re thrilled to be using fresh ingredients because the quality of the purchased stuff out of the bag/can is really different. And it’s worth noting that the local tomatoes were grown in 2009, a very wet, cool summer that tended to make tomatoes blander and more watery than usual, so we had a tough-weather-condition local ingredient going up against one of the highest-end canned version.

Goat vs. Venison, red-wine marinated
Goat: 1.71
Venison: 3.14

Clear win for goat, mostly based on texture. Most people didn’t see much difference in flavor, but the venison was tougher while the goat was tender and flaky. This was all the more impressive since I couldn’t match the cuts exactly from what we had left in the freezer, and used venison backstrap and goat “ham”. So even with a higher-quality cut of venison, it still lost badly. Of course, we were still comparing a young, milk-fed kid with a mature wild deer, but I wasn’t expecting that spread.

Green beans
Farm-frozen: 1.83
Store frozen: 2.83

This was a bit unfair, as we used frozen Fin de Bagnol beans, our top-quality filet beans, as against a standard frozen organic bean. The appearance difference was quite obvious, and may have influenced the results. Still, folks were pretty clear on this.

Goat vs. Venison, fresh-ground burgers
Goat: 1.71
Venison 3.00

Again a clear win for goat, with similar comments on a superior texture for goat though the flavor wasn’t much different. Clearly we need to raise more meat goats and figure out how to have the meat legally and economically butchered. Goat is certainly underrated in the US, especially young, milk-fed kid like this one was.

Sweet potatoes (roasted fries)
Farm-stored: 2.17
Storebought: 1.83

A narrow win for the store (conventional from Hy-Vee). I really couldn’t tell the difference, and most folks felt it was subtle. Ours probably haven’t been stored at ideal conditions, and depending on where Hy-Vee got theirs, they might have been fresher, though they looked very beat-up surficially. But the good news is, we don’t grow sweet potatoes for sale, just for us, and if we can match the quality available elsewhere in a far cleaner and known form, we’re happy with that in our pantry.

Dried tomatoes
Farm dehydrator-dried: 1.57
Store sun-dried: 2.71

This was pretty clear. Ours were sweeter with a better texture, while the storebought ones were darker and tougher with an odder flavor. We’re not allowed to make and sell these without a certified kitchen, but customers can certainly make their own to get a better quality than store-bought.

Chamomile tea
Farm-dried whole blossoms:
Storebought tea bags:

We were all a bit punchy by the end, possibly from the effects of the wine, beer, and hard cider tastings brought by guests, so data didn’t get recorded for the tea. However, the discussion made it clear that everyone though the farm tea was far more flavorful and good. So we need to find out whether we can legally dry blossoms for sale, and whether we can set a rational price.

Overall, having done our best to buy the highest-quality equivalents, prepare everything simultaneously/equally, serve them blind in random order, and get reasonably accurate ratings, we’re happy with how this turned out. The corn/polenta was the biggest surprise, but given the large number of customers who have told us how much they like our cornmeal overall, and our experiences with the raw/early cooked store stuff, we’re probably not going to worry too much about that test. Fascinating that goat was far more popular than venison, but at least it’s theoretically possible for us to legally sell goat.

Overall, our farm products held their own or beat the best store-available alternative, in March when they’ve all been frozen or stored for many months, and with far more known growing, handling, and harvest methods with far fewer food miles on them. If we can match the stores even in winter, I know what we can do in summer (and what customers can do with our products).

I think Scott Rowson asked the best question: “So the results seem to show that the best-scoring items are the ones you can’t sell, like cheese. How do you feel about that?”. Well, I feel pretty annoyed. If the cheese we make in our home can outrank some very high-end commercially available cheddars, what justification is there for keeping it illegal? I’d happily submit every round to a health department swab test to prove it’s “clean”, whatever that means, and I think everyone at this meal would happily have paid for some. Good thing we have food safety laws to protect us from ourselves.

Grousing aside, it was a great day, with great people, just a really nice fun atmosphere. Thanks to everyone who took a big chunk out of their day to join us.

2 thoughts on “Tasting meal results

  1. I don't for the life of me know how you got the energy to type this after the day we had, but more power to you! I want to mention too that Katy is a serious cheese person and her eyes lit up when she ate your cheddar. Seriously good work there. Thanks for having us, it was a real treat and overall a very cool thing to be a part of.

  2. Well, you all left around five. We fell asleep on the couch to Finding Nemo, then woke up to go do animal chores at dusk. After that we were fine for a few hours, in which we cleaned up and wrote the summary post, then had "dinner" of leftovers in bed around 10, then crashed. No problem!I pretty much never talked to Katy, so my apologies to her. We're real good at hosting events in which we're busy all the time. Looking forward to a chance to chat more with her. Glad the cheese was approved; we'll look forward to teaching you to make your own this summer.