If each animal could be photographed and uniquely identified many times each day, the science of ecology and population biology, together with the resource management, biodiversity, and conservation decisions that depend on this science, could be dramatically improved.
IBEIS is a large autonomous computational system that starts from image collections and progresses all the way to answering ecological and conservation queries, such as population sizes, species distributions and interactions, and movement patterns. The images are taken by field scientists, tourists, and incidental photographers, and are gathered from camera traps and autonomous vehicles. IBEIS can detect various species of animals in those images and identify individual animals of most striped, spotted, wrinkled or notched species. It stores the information about who the animals are, where they are and when they are there in a database and provides query tools to that data for scientists and curious people to find out what those animals are doing and why they are doing it.
Tuesday, July 22, 2014
The Drop in Safari Tourism
We felt the impact of the drop in tourism when we stopped at our favorite curio shop on the way back to Nairobi. Unlike our earlier visit, they seemed almost desperate for our business. We didn't have the heart to push extra hard on the prices during our negotiations. Still we are heading home with some beautiful "stuff".
More Poaching!!
When we drove though Ol Pejeta Friday morning, we watched low flying planes and then saw a helicopter at conservancy headquarters. We did not know what had happened until later. The senseless attack, driven by someone's false notion of an aphrodisiac, is a complete waste. Even though the IBEIS project does not contribute directly (at least not at this point) to the anti-poaching efforts, the recent killings of rhinos Ol Pejeta and nearby Ol Jogi (a few days before we arrived) added a sense of immediacy to the problems Kenya and other nearby countries face in their conservation efforts.
Monday, July 21, 2014
Connections made in Kenya
And then it rained. The crew had to get back to Ol Pej and Blair did a sisyphean job of driving everybody back in the rain over clay roads that when wet instantly become trecherous ice skating ditches. 20km can take over 2 hours!
July 2014 Ol Pejeta IBEIS Team
Challenges of Working in the Bush
Yet, somehow things are getting done. Go team!
Saturday, July 19, 2014
A good luck charm
I did a little bragging about my good luck to the latest group, and after the drive we had this morning I'm beginning to think I should visit the casinos when I get home! Our first stop was the marsh where we found twelve lions dining on a breakfast of freshly killed zebra in the golden morning light. And they were right on the road so we got close!
Really close.
This group was into birds as well, and we checked off a few new species and got a great view of this harrier hawk raiding other birds' nest in a dead tree:
Well I'll be sure to brag about this drive to my next potential group of tourists - hopefully I can continue bringing great sightings to them, and good data to IBEIS!
The new face of IBEIS
Well, the "why" is simple: perception and understanding are inexorably linked to the presentation of information: what you say is affected by how you say it. To dismiss design as decorative is to pretend that rhetoric and oration are just flowery writing and a lot of hot air, and while good design shouldn't mask bad science, good science can certainly be hindered by bad design. To wit, if we didn't judge all those proverbial books by their covers, then a lot of very talented book designers would be out of work.
Thankfully, IBEIS is different. The lofty research- and conservation-oriented goals of the project require an enthusiastic and engaged public, and a public-facing project needs a consumer-quality identity. Enter yours truly.
As an illustrator and designer, I find I'm less concerned with any single particular execution than I am with the needs of a given project. Sometimes I get to draw, and sometimes I get to write code, but telling the right story is always paramount. By virtue of a certain non-work choice—my engagement to Dr. Blair A. Roberts, whom you've already met—my professional life has led me to some very interesting places. This isn't my first lengthy trip to Kenya, though my second stint in this beautiful country is slightly more work-oriented than last time.
The logo, the foundation of any project's visual identity, needed to allude to a few key components of IBEIS: it needed to refer to some of the species that comprise the IBEIS Lite test group; it needed to show some link to the analysis of digital images; and it needed to catch the eye and excite the public. I'm sure you won't begrudge my bias if I feel we hit the mark on all three counts with this:
Martial eagle
A Moment of Two Transitions
Clara, Jon, Jason and Blair head toward Ol Pejeta HQ and the server room. |
Friday, July 18, 2014
My Experience in Kenya so Far.
to mind. My name is Jon Crall I'm an introverted computer scientist with an
obsessive need of consistency and a phobia of new foods. I also wrote the first
core recognition algorithm for IBEIS. Needless to say, I was nervous when asked
to travel to Kenya to deploy a prototype of IBEIS in the field.
This blog post will not be a story of a spontaneous transformation into an
extrovert who loves to travel and try new things. It will be honest account of
my feelings and experiences here. But hopefully it wont be a sad story of doom
and gloom. So often people box themselves a false dichotomy: They assume that
somebody else is not experiencing or enjoying something as wholeheartedly as
themselves then they must be at the opposite end of the extreme. I was not
dragged hear kicking and screaming. I chose to come because I truly care about
this project and what it is capable of becoming. I also knew that if I never
left my comfort zone (approximately a 42 mile radius around the city of Albany, New
York) then I would never become the great scientist that I someday hope to be.
Lets start off with something I truly love: information. If you ask Leonard
Susskind -- the physicist who challenged Steven Hawking and won -- , he'll tell
you that its the primitive building block of the universe. How fortuitous that I
find myself in a field where the bit is the fundamental unit. How lucky that --
as a cashier at Price Chopper -- I found myself with the gall to ask Naomii
Hoffman (the wife of the Vice President of Kitware) for an internship when she
came to my register. Its still mind-boggling to me how through Kitware I wind up
being accepted to the PhD program at RPI and working with Chuck Stewart.
While working with Chuck I delved into the world of Computer Vision -- the
science of using image and video to elicit and use information about the word. I
found myself with the challenge of recognizing individual zebras from
photographs. Over the next 2-3 years I learned enough to create HotSpotter, an
algorithm and primitive user-interface for identifying patterned animals,
specifically zebras. I wont talk much about the technical details (although I
really really want to); I'll just say that the algorithm was a success.
Due to the success of HotSpotter we decided to set our sights on a much more
ambitious goal: automagically build a database of tens of thousands (a baseline
number) of wildlife images and tag: where the animal is in the image (the
detection problem), who the animal is (the recognition problem; my problem), and
where and when the image was taken (easily captured by camera sensors). Given
these primitives it becomes possible to rapidly answer complex biological
questions.
I'm afraid the words "complex" and "rapidly" do not begin to do justice to how
big these questions can be and how fast we could answer them. I suppose those
who imagined and built the internet must have had a similar feeling, either that
or I have a bit of an ego, or perhaps I'm boxing myself into a false dichotomy.
This software has profound implications for conservation science. Not only will
it help the biologists do their job, but through the help of Wildbook, maybe we
can start to shift the public opinion into one which cares enough to take action
about biological diversity, ecosystem stability, and respects and acknowledges
the conciousness which every organism with a well-developed neurological
(/information processing) system must have.
Maybe now you can begin to understand why I chose to go to Kenya and leave the
comfort of 24/7 electricity, internet, abundance of plain un-intimidating food,
and the often overlooked "first world" septic system.
I hope in the context of this primer the reader can begin to appreciate the
words I have carefully chosen to describe my experience here.
For a modern day U.S citizen to find themselves in Kenya he/she must first go
through a 20 hour trial where you are exposed to excess radiation, deprived of
sleep, packed into a sardine can, and hurled through the sky. Upon landing
fingerprints are registered and you move through armed guards into a sea of taxi
drivers competing for clients.
I am extremely grateful for the buffer of the first night. We stayed at the
Country Lodge Hotel where we were given accommodations similar to those I'm
privileged to be accustomed to. I felt safe as we were in a gated area and I had
already injected with a plethora of immunizations. The only precaution were were
given was not to drink the tap water. Apparently the most common mistake a new
traveller in Kenya makes is brushing his/her teeth with the tap water. Naturally
with this in mind and a strong fear of getting sick, I panicked as I observed
myself do exactly that under the influence of muscle memory. Having done the
deed, I recited the comforting mantra: "Don't Panic" as I rinsed my mouth with
bottled water and fell asleep.
The morning brought a pleasant surprise: food I was comfortable with. I had
packed about 60 cliff bars, dried fruit, and turkey jerky in preparation for
being trapped in a novel food landscape (Here I will remind the reader that I
have a food phobia. I will also add that it is my biggest phobia, the second
being bees/wasps/yellow-black creatures with stingers). I feasted on scrambled
eggs, toast, bacon, and fresh mango (a delicacy that is impossible to find in
the States. Mangos do not travel well).
We (Chuck, Jason, Jon, and I) took the 4 hour drive from Nairobi, through the
equators, to Ol' Pejeta Conservancy. In a state of apprehension I reminded
myself to keep myself calm and relatively open to experience and notice the
landscape, architecture, and people. The most striking feature was the Kenyan
soil, which was painted red with clay. It was so pervasive that even the plants
had a dark red tinge to them. The buildings were being constructed rapidly and
supported by a winding scaffolding made of trees. Apart from the red soil the
most distinctive feature I noticed was how twisted the trees were.
We passed many gated schools and universities, farmers, goats scaling the faces
of clay cliffs, fields of maize, and a large number of pedestrians. This was
expected. I've seen many pictures of Kenya (I've actually seen more than most
people due to the nature of my work). What was not expected was that the
pedestrians never stopped. In the states you would never find someone walking
alongside of a major highway, but here there was always at least someone walking
or biking somewhere along the road.
Upon arriving at the conservancy our car was searched by guards and dogs.
Poaching a serious problem here. Its disgusting how many Rhinos are murdered for
nothing more than a horn made out of the same material as your toenails:
Keratin. We passed the inspection and were allowed to drive to the research
center.
At the research center it turned out that the conservancy staff was not
expecting us. Despite this we were served pizza for lunch. I ate a cliff bar.
We sat at the research center and waited for the staff to confirm that we were
researchers and actually had a reason to be there.
Later, after bureaucracy happened, they set up a camp site for us along a river.
We were given tents and cots which were about 3 inches to short for a person
standing 6'3". I learned something new about myself: I was still afraid of the
dark. I had thought I had moved past this years ago. Maybe it wasn't the dark.
Perhaps it was simply the more universal, ill-defined, and permeating fear of
the unknown. That and spiders. The latrine had a lot of spiders.
I've made it through three nights so far. They are by far the hardest part for
me. At the equator the sun rises and falls very quickly, so even though you get
6 hours of punctual light, there is a tangible anxiety in the last lit hour, and
a recovery period in the first. The silver lining is that the lack of light
pollution and the opportunity to view of our galaxy with the naked eye.
The days are far more pleasant than I had expected. The small tube of SPF45 that
I bought seems to be enough even for my ginger skin, the heat is far from
brutal, and the bugs don't come out until dusk. I'll leave the detailed
descriptions of the wildlife to the other bloggers, but I will say there is a
certain je ne sais quoi in seeing a savannah with elephants, zebras, lions,
etc... with your own eyes unfiltered by a window or lens. I should also note
that the soil in Ol' Pejeta is not red (lest the reader get the false impression
that the entire ground of Kenya is clay), but the more familiar brownish-yellow
color normally associated with savannah.
I'm worried that its almost too obvious that I don't share quite the same level
of enthusiasm as the other researchers here. Its difficult to find a way to say:
"I'm really really uncomfortable and nervous. This may not be my idea of a
vacation, but I really do care about this project and want to be part of this
team." Maybe its as easy as that, but it seems so much easier to be quiet and
agreeable in the face of so much unmatchable excitement.
Yet, the trip is still not over. In the spirit of science I'll do my best to
reserve judgement until all the evidence is in, although I suspect my hypothesis
that I'm not much of a traveller will be confirmed.
So, I'm anxious. Maybe irrationally so, but what I'm feeling is real and demands
to be felt. But its clear to me that I care about this project (one does not
have a nervous breakdown about something he/she does not care about) and want to
see this trip through. Will I want to come back? Well, re-read the last
paragraph and ask me on the 23rd. For now I'm focusing on leaving the best piece
of software I can muster, reconciling a sense of perfectionism with a finite
lifespan, and counting the days until I can return home to my person I have
pair-bonded with.
Herds of data
Thursday, July 17, 2014
Wild dogs, rhino charge, and white spaces
So until this point the most exciting event of the day was being charged by a rhino about half an hour prior, at Ol Pej almost out the gate. We were watching a typical ate afternoon scene: impala, zebras, hartebeest, all alert for some reason. There are three jackals near by but we don’t think that all those animals are concerned about them so we keep trying to find whether there is another predator. Nope, doesn’t seem so. We are so absorbed by the jackals that we don’t notice for a while that there is a family of rhinos a bit further away. We drive to them: a big male, two females and a calf. The male is guarding the family, alert. We are taking pictures of the rhinos when suddenly the males decides to charge. Fast, gaining speed. Jakson quickly responded by starting the car and accelerating away. The rhino kept up for a while but finally stopped. Our hearts were punding.
Until this point the most exciting part has been the meeting with Malcolm Brew and Sid Roberts from Microsoft 4Afrika White Spaces Mawingu project. Looks like this project may be able to provide the bandwidth necessary for the server to go on Azure cloud and to be accessible remotely.
The server
We turned it on yesterday. Drum roll please. Lights on. The tense few minutes of waiting for the Linux interface to come up: “Initializing:… 0%” And then the scroll of lines across the screen. The jet engine taking off sound of the cores starting. Success!
Now we have a server in our attic.
Just one day....
. Photographing kudus up close before leaving MPala for Ol Pejeta.
. Testing and debugging the IBEIS image analysis software.
. Meeting a German television team interested in doing an IBEIS segment for a program aimed at 9-14 year olds.
. Intense debates about program semantics and the supporting graphical user interface during a fire-cooked lunch by the river.
. Starting up (successfully) a 20 processor 128 GB memory server - one that has been enroute from Chicago for three weeks - in a new room on the second floor of a (big) garage-like building whose stairway and hallways are cement and whose offices open to the outdoors.
. Visiting the Jane Goodall chimp enclosure and going face-to-face with our closest evolutionary relatives.
. Debating alternative approaches to data modeling for queries about ecology and animal population dynamics.
. Singing Happy Birthday (belated) to team-member Jon Crall.
. Stopping to watch and photograph a beautiful female lion on the way home.
. Sipping wine and scotch over a three-course meal, including a salad made by Githae (the chef at Mpala Ranch) from the cook book Lou Ann sent with me.
. Reviewing progress and strategizing for tomorrow.
What more can one ask of a day?
An introduction and a challenge
I have several tasks while I'm here. First and foremost I am responsible for riding along with tourists on their game drives. The purpose is to document what tourists take photos of, and what they miss. This will help us to understand the sampling biases inherent in a tourist-generated data set. I've gone along with a couple groups so far and my first impression is that there are a couple different types of tourists. First-timers (those who are on their first trip to Africa, or their first visit to Ol Pejeta) are understandably taken with the sheer abundance of game and the beauty of the wildlife here. They are happy to stop at every zebra, impala and gazelle and will take hundreds of photos on a 90 minute game drive. I can relate - I see shades of myself on my first trip to Kenya in 2008. The other type are the veterans, those who have travelled Africa and Kenya extensively, or who have returned to Ol Pejeta for their second or third visit. These veterans are more particular about their photo opportunities. Predators and other rare species are their targets, although a zebra or antelope may serve as a subject if the scenery is particularly beautiful, the light particularly golden, or if the animal is engaged in a behavior or interaction they have never seen before. These are just my first impressions, but so far I think they will be borne out in the data.
A lioness spotted on one of the tourist ride-alongs. It may be possible to use the spotting on lions' legs and bellies to individually identify lions |
Captured while setting up a camera trap on Pelican Dam. |
Elephants trigger a camera trap by the Ewaso River. |
For a given female animal (let's say a giraffe), I would like to know the identities of all of her offspring, along with their date of birth (or more realistically, the date they were first sighted). This will tell me how many offspring this giraffe has produced, as well as the age of her offspring. For each of these offspring giraffe, I also want the date of their last sighting as a subadult with their mother (which will tell me when they dispersed from their natal group), as well as their last sighting overall. If this last overall sighting is more than a year ago, I would like to know the frequency of sightings prior to that. If an animal has not been sighted in more than a year, it could be that they are deceased or have otherwise left the study population. If I know the frequency of sightings prior to the final sighting, I will be able to tell if a year gap in sighting is abnormal or not. If a year gap is abnormal, and many times longer than the usual gap, I might reasonably conclude that that animal is no longer present in my study population.
So there's my challenge. When it's met (as I'm sure it will be), I'll be ready with another!
Wednesday, July 16, 2014
IBEIS Server at Ol Pajeta
However, now everybody is excited about it... arguably the most powerful machine in East Africa with its 40 Xeon cores, 128 GB of RAM and 24 TB of disk storage. Surely more than enough to cope with the huge number of animal images we expect to collect and process daily as a part of the IBEIS project.
Monday, July 14, 2014
Introduction to IBEIS
Our team of faculty, graduate students, and professionals from Princeton, Rensselaer Polytechnic Institute, University of Illinois-Chicago, and WildMe, working in biology, ecology, computer science and software engineering, has gathered work at Ol Pejeta Conservancy and Mpala Research Center in the Laikipia region of Kenya to install, test and refine a (very) preliminary version of IBEIS, an Image-Based Ecological Information System.
What does this mean?
. Pictures! Tourists on safari take lots and lots of pictures. So do scientists and technicians. Now mix in automatic systems like GoPros mounted on tour vehicles and (eventually) UAVs with cameras on them. The result is many thousands of pictures a day, and at just one location.
. Algorithms and software. There are too many pictures and too many animals to process the data by hand. Instead, we are developing algorithms to find animals of certain species in the images - zebras and giraffes so far, with elephants and others soon to follow - and algorithms to identify individual animals if they have been seen before - works with almost any species with stripes or spots - or to decide which ones are new. We have built a software system around these algorithms. All are very much works in progress, but we are testing them here "on the ground" in Kenya and learning first-hand about how they might be used.
. Science. Using the pictures, IBEIS will determine who the animals are, where they are, and when they are there. From this, the scientists can determine what they are doing and why they are doing it. With so many pictures, we should obtain a great deal of detailed information... Who? When? Where? What? Why? That is our mantra
. Tourists. Those who contribute pictures will quickly learn about the animals they photographed through the magic of IBEIS's detection and identification software. Using the version of IBEIS we are installing, tourists will be given simple life histories of "their" animals by the next morning. In the future, the feedback will be almost immediate. The goal is an enhanced tourist experience, leading to a long term connection between tourists and the animals they saw, the wildlife parks they visiting, and ideally conservation in general.
. The Team. We are working together at many different locations, including New York, New Jersey, Illinois, Oregon and, of course, Kenya. We need to get to know each other, form a cohesive team,
understand our purpose, and push toward our goals. It doesn't hurt that we are doing this in Kenya, with baboons and vervet monkeys lurking around the corners, and giraffes, elephants, zebras, rhinos,
impala, warthogs, buffalo, fish eagles, secretary birds, gazelles, and maybe lions, leopards and cheetahs just a game drive away. Talk about incentive for doing well! Just this afternoon driving back from Ol Pejeta to Mpala, I saw an infant plains zebra, zebras mating, rhinos fighting over a potential mate, and my first kudus and elands!
So, with this introduction, Tanya and I will invite the team members to write by contributing quick thoughts, short vignettes or longer descriptions.
We are excited about IBEIS, thankful to our universities, to Microsoft Research and East Asia, to Ol Pejeta Conservancy, to the Mpala Research Center, to private donors who have helped us get started, and to the US National Science Foundation. And, we are anxious to get started!
Sunday, July 13, 2014
Microsoft Faculty Summit online
A sad welcome to Kenya
Two armed gangs killed four rhinoceroses for their horns in rural Kenya this week in possibly the worst rhino poaching incident in the country in more than 25 years, the spokesman for Kenya Wildlife Service said Friday. Poaching across sub-Saharan Africa is on the rise as armed criminal gangs kill elephants for tusks and rhinos for horns, usually to be shipped to Asia for use in ornaments and medicines. The poaching on Wednesday night took place at the private Ol Jogi ranch near Nanyuki, about 120 miles north of Nairobi. Paul Muya, a spokesman for the wildlife group, said that the poachers escaped with three of the animals’ eight horns. The killings raise the number of rhinos poached in Kenya so far this year to 22, which leaves just 1,037 rhinos still roaming private wildlife conservancies and national parks, Mr. Muya said.Ol Jogi shares a border with Mpala Research Centre, our main research base and is in the same wildlife group as Ol Pejeta. This is after Kenyan government took away one of the anti-poaching tools: Government bans drone use to fight poaching in Ol Pejeta Se we hope that part of the many uses of IBEIS will be to be keeping the many eyes on the endangered animals though images.