From Maslab 2006

Maslab teams
Team 1 · 2 · 3 · 4 · 5 · 6 · 7 · 8 · 9 · 10 · 11 · 12 · 13 · 15
Team Seven's Journal · Paper

Contents 1 Mechanical Design 1.1 Ball Collection and Deposit Mechanism 1.2 Sensors 2 Software Architecture 2.1 Multi-Threading 2.1.1 Robot Control (Head) 2.1.2 Image Capture Thread 2.1.3 Image Processing Thread 2.1.4 Velocity Control Thread 2.1.5 Bump Sensor Thread 2.2 Robot Movements 2.2.1 Stopping 2.2.2 Turning in place 2.2.3 Drifting 2.2.4 Releasing Balls 3 Image Processing 3.1 Overview 3.2 RGB 3.3 Vertical Scanning Is Easier 3.4 Multiple Object Recognition 4 Artificial Intelligence 4.1 Basic Components 4.1.1 Wall Following 4.1.2 Ball Following 4.1.3 Getting in Range 4.1.4 Aligning to Score 4.2 Complex Algorithms Using the aforementioned Basic Components 5 Controls 6 Contest Evaluation

Mechanical Design

How we built the contraption.

Ball Collection and Deposit Mechanism

The cornerstone of our design was a conveyor belt that lifted balls into a chute from which they could be released independently. Cupboard liner proved to be the ideal material for the belt since its unique surface gripped the balls as they were propelled up a ramp. The belt was laid over two aluminum rollers covered with foam pipe insulation, which allowed them to deform and suck the balls in as the robot drove over them. The rollers were 6 inches wide to allow for significant error in our aligment and were run continuously to maximize our chances of collecting a ball. After being sucked in by the roller, the belt would roll the balls up a sheet aluminum ramp covered in liner. The entire mechanism was housed between two square pieces of plywood and was powered by a drive motor attached to the top roller. Once a ball had been lifted up the ramp by the conveyor, it rolled down a chute and was stopped by a servo-controlled gate. The gate was made of sheet aluminum and was shaped in the form of an L in order to allow the gate to be opened and closed quickly enough to only allow one ball to be released at a time. This mechanism worked beautifully during testing, but we didn't get a chance to show it off in the actual contest until the exhibition round. Once a ball was released, it traveled down the remaining part of the chute and rolled fairly straight for 3-4 feet. The ball trajectory was so predictable that it could be overlaid onto our processed image to aid in aiming at the goal.

Sensors

Our design used three short range IR sensors to avoid obstacles and allow for wall following on the left side of the robot. One sensor was aimed forward while the others were angled 30 degrees forward from both sides of the robot. The camera was mounted low and directly over the conveyor's first roller. I was slighlty angled downwards to make it easier for the robot to engage and align with a ball in its path. Since our strategy required our robot to be able to count the number of balls it had picked up, whisker sensors were placed at the top off the conveyor. When a ball rolled off the conveyor it flipped a small brass rod that activated the sensors and added another ball to our counter. This method proved much more reliable that the alternative of assuming that every ball that had been engaged had indeed been caught. Driving straight was another performance requirement that was met using the sensors provided by the Maslab staff. High resolution encoders were mounted on both wheels and a velocity controller was used to both help the robot drive straight and also detect when it was stuck.

Software Architecture

How we gave the contraption a mind of its own.

Multi-Threading

Robot Control (Head)

This is the main controller for the robot. It would specify what state we were currently in and would call the desired actions (wall follow, ball follow, score.) This is where we would initiate, start and eventually stop all the threads as well as control what actions we want to take given the running time: if less than three minutes had passed, the robot only released two balls per goal, otherwise it would release the number of balls it thought it had +2 in case we didn't count some balls. This would not be very costly since we estimated the release of a ball to be approximately 2.7 seconds.

Image Capture Thread

The Image Capture Thread had only one task: to call Camera.capture as many times as possible while running. We were told that the capture method had a sleep inside it, making this capture method relatively slow (compared to the processing) so that we should split these two so that the Camera tries to capture as much as possible and in the time it is waiting to capture, we are able to process the images.

Image Processing Thread

The Image Processing Thread took the latest picture in the array of captured images and processed it by extracting the relevant information and setting variables that would be accessed by other methods and classes. It would also publish every 100 images to the BotClient so that the audience (and our debugging team) would see what the bot is seeing. This was particularly useful when we were extracting the features from the image because since we use RGB instead of HSV, it is a little harder to tell if a given combination is Yellow or Red. We strongly recommend using BotClient early on so that you can see in real time what the bot is seeing.

Velocity Control Thread

The Velocity Control Thread would take readings from the encoders and the desired speeds the robot wanted to be in and, using a PID controller, would do all it could to get the veolcities to something reasonably close to what we actually wanted. This worked pretty well as we really never had any speed issues when we dealt with low batteries and similar problems that would show an adverse side effect had we simply set fixed values to the motors.

Bump Sensor Thread

The Bump Sensor Thread was in charge of waiting for a click of the bump sensors so that we knew how many balls the robot had picked up. Since there really was no way to tell if we had picked up a ball or not (since the camera did not have such a big spectrum,) this was the next best thing. It would sometimes miss-fire when a ball got stuck on the bump sensor, so we had to implement a way of telling the counter that if the reading was true the second before and it is true again now, it probably means the ball is still there, so don't add another ball to the counter. This thread would sleep for 300ms after checking if a ball had passed by; this seemed like a really long time, but it worked perfectly.

Robot Movements

The title of this sections seems a little misleading: we simply mean how did we get the robot to move. What abstractions did we use to get the piece of metal/wood/plastic to move?

Stopping

Well that's easy, set both motors to 0. NEXT!

Turning in place

In order to turn in place, we had two different approaches. We originally had a method that would turn one motor forward at full speed and one motor backward at full speed, sleep it for a certain TURN_TIME and then set both motors to 0. This worked, but would sometimes overshoot if we changed the TURN_TIME and would sometimes be slow of a turn if the motor acceleration was not high enough. So we decided to implement a turn that simply set one motor to a TURN_SPEED and the other motor to -TURN_SPEED. No stopping involved, we would simply wait for the next command which would change the motor speeds.

Drifting

In order to drift in one direction, we followed the same idea as the second implementation of turning: set each motor to a desired speed until the next command was given. We had to implement different kinds of drifting since sometimes you want to drift sharply (almost like a turn in place,) but sometimes you want to go relatively straight, giving preference to one side over the other. This wasn't too hard, we simply tested different speeds and for the straighter drift, made both motors go at similar speeds, but one should definitely be faster in order to get the drifting effect. For the sharper drift, we had one of the motors go at the lowest possible speed so that the robot still moved (~0.4 - this seems high, but remember that the motors need to cary a lot of weight and break the static friction barrier,) while the other motor went at a speed we declared CURRENT_TOP_SPEED. This fixed constant would be changed, but we always knew it would be at least 0.2 more than the lowest movement speed, so it guaranteed us a drift either way. Towards the end, we were so tired of playing with the drift speeds, that we implemented a method that would take as an input the given speed we wanted for the faster motor, that way we wouldn't have to go back and change the fields in our RobotMove class.

Releasing Balls

This was a different class that simply dealt with the gate servo. We tested to see what angles would produce an "closed" state (no balls could go through) and an "open" state (once the servo comes to this angle from the closed position, it would guarantee us that if there were any balls in our shute, at least one would be released.) We had a pretty high accuracy with this (at most two times out of the hundereds of times we tried this,) so we were pretty confident that it would only release one ball at a time when going from closed to open. Worst case scenario, we drop four balls into a goal instead of two; what's that, 14 points instead of 10, potentially 20? Not too shabby.

Image Processing

How pixels become balls and goals.

Overview

Image processing plays a big role in Maslab. Without the camera your robot will be rendered usless; unable to recognize balls, barcodes, and goals. There are several ways to approach this complex task, but we will discuss how we analyzed each pixel, scanned each image, recognized multiple objects, deciphered barcodes, and found the center of the goals.

RGB

It was by accident that our team ended up analyzing pixels using RGB values rather than dealing with the HSV spectrum. We somehow got a hold of two different ImageProcessingTutorial.java files; one using RGB and the other HSV. After spending hours trying to figure out why all of a sudden we no longer could correctly recognize colors using HSV thresholds, we realized that our IPT.java file had been swapped and we were now dealing with RGB values. Frustrated by the lost time we decided to simply keep on working in the RGB domain. Little did we know how much more effective and reliable this "accident" would turn out to be. For starters, we no longer needed to waste any computation power converting rgb values into hsv and consequently, we didn't have to "paint" each pixel into the hsv spectrum. Our team never had problems recognizing different colors under various lighting conditions, nor did we ever have to continuously deal with the pain of tweaking our thresholds. The way I see it is if there is no advantage to working in the HSV spectrum, why bother.

Vertical Scanning Is Easier

Scanning an image row by row is more efficient because pixels, when read from an image in Java, are stored in a row-major-order. Meaning that once we access a location in memory, surrounding values can be accessed quicker because they are probably already in the cache. Scanning horizontally, however, made the task of recognizing multiple objects a daunting one; especially for a mechanical engineer. The team decided to test how much more efficient scanning horizontally was versus scanning vertically, and to see whether the computation trade-off was worth the complex coding. Reading through an image row by row and column by column took 20ms and 30ms, respectively. After a quiet sigh of relief, we all agreed that the extra 10ms was more than acceptable and made our programming a lot simpler.

Multiple Object Recognition

Being able to recognize multiple objects of the same class is a key task. By doing so, for example, it is then trivial to distinguish between multiple red balls and know which ball is closest to your robot. The main idea behind our multiple object recognition algorithm consists of making virtual boxes around regions where we have detected blobs of importance (red == balls, green/black == barcode, etc...) and then analyzing each box individually. Imagine a partially filled and enclosed tic-tac-toe game.

The first step is to draw a box encompassing all the blobs. This is easily done by remembering the min and max x/y values of where you have detected a desired color. The second step consists of scanning this box horizontally; essentially drawing vertical lines that seperate the multiple blobs. Analogously, the third step consists of scanning the entire box vertically; essentially drawing horizontal lines that seperate the multiple blobs. The result is a grid whose elements are either empty or contain a blob. At this point it is a matter of just analyzing each grid element. Is the most efficient way? Probably not, but we managed to process images at a rate of 15-20 fps which proved to be very effective.

Artificial Intelligence

The best thing about Artificial Intelligence, in my opinion, is that you're not always aware of the fact that you are using Artificial Intelligence: "it all seems natural to me!" Once you see that an innanimate object is performing a relatively complex task, you then realize there must be some AI involved.

Basic Components

We decided to start out with the basics: Wall following, Ball following, Getting close to a goal, and finally aligning itself in order to score.

Wall Following

Our wall follow algorithm seemed to be the best one working, and we have to thank simplicity for that. Our strategy was as follows (in pseudo code): while(wall following){ 1) If you have an obstacle up ahead, turn right 2) If you have an obstacle on the left, drift right 3) If you have an obstacle on the right, drift left 4) If you are aware there is something on the left, but are not in danger of running into it, go straight 5) Drift left rinse, lather, reapeat! }

Ball Following

This was our main algorithm for picking up balls. Basically, once we had a ball in view, we divided the image into three sections: left, center, right. The center had a threshold of about 20 pixels left and right of the actual center which represented the width of the conveyor belt (if the ball was in this section, all we had to do was go forward and the conveyor belt would surely pick up the ball.) If the ball was on the left portion, we would drift left until the ball entered one of the other two sections of the image. Similarly, if the ball was on the right portion, the robot would drift right until the ball entered one of the other two sections of the image. This would make the robot act a little erratic, but after lowering the speeds and increasing sleep times here and there, we rarely missed a ball that came in sight.

Getting in Range

Once we had balls to shoot and a goal in view, we had to ensure we were close enough to start targeting. For this we used the yellow area as an indicator of distance. We did some testing and found that if the yellow area in the image was above a certain number, we would, at some angle, have an adequate shooting position. This was our downfall on the final competition since we did not implement analyzing multiple goals and when we saw so much yellow (since there were two goals right next to each other,) the robot believed it was close enough to shoot and began searching for an adequate shooting angle.

Aligning to Score

Originally, we thought of using a crosshair (think Goldeneye and Halo) in order to find a good shooting angle. This was fine at first, in our imaginary world were robots actually do what they are supposed to and 1.5 is really 1.5 and not 0.9 or 2.0; once we actually got to some testing, we realized a trajectory or path would be a much better way to target. Since we originally had thought of having the shute come down somewhere through the middle of the robot, we had used a trapezoid as our trajectory. Once this idea was thrown out the window, we simply rotated and streched the trapezoid a bit and we saw that the ball went through our projected path about 95% of the time. We had to limit the range of the trapezoid since after certain distance, the ball would become unpredictable and would sometimes drift left, but sometimes drift right. This was no problem since our shute allowed us to shoot from approximately 2 feet with some certainty of where the ball would go. Our algorithm consisted of checking if the center of the goal was in the projected path and that there was no yellow pixels inside the projected path. If not, it would turn left in place for a small angle and recalculate the values. Since this was a very important task, we decided to make the robot check twice if it thought it was capable of scoring by refreshing the image and checking again that it was still capable of shooting. This is something we strongly suggest for teams who wish to use some sort of shute that requires aligning.

Complex Algorithms Using the aforementioned Basic Components

They're not really Complext Algorithms once we have these basic components. The idea was to wall follow until the robot either saw a ball or saw a goal. If it saw balls, it would go pick up the closest one; if it saw a goal, it would check if it had any balls, and if it did, it would get close enough to shoot and then it would try to align itself and finally shoot. This is very similar to the idea of a Complex Learner composed of many simple learners. For more on this, take 6.034.

Controls

Our primary strategy was to wall follow until we saw a ball and then go after the ball. This meant that our dependence on extremely accurate proportional derivative motor controllers was limited. In our case a slight curve was good because it allowed the camera to sweep more of the playing field. When the IR sensors indicated that we were to far from the wall they would signal the robot to curve to the left and narrow the gap.

The only precise controller that needed to be developed was the alignment controller. The mechanical design of our robot called for the chute to exit on the right of the robot’s center, offset from the center by three inches. After the robot determined that it was the desired distance from the goal, 18inches, it turned to center the chutes’ deployment path with the center of the goal. This process was published to the bot client and unfortunately we were not able to demonstrate it for the audience but they would have been impressed as the robot overlaid the trajectory of the ball with the processed image and showed it aligned for the center of the goal. In practice this system worked remarkably well and the entire team was impressed with how little trouble shooting and error fixing we had to do with this system. While we set up our bot to shoot from 18 inches away for the competition, during testing we were able to score from 4 feet away.

It is worth mentioning, even though we ended up abandoning this strategy, that early on we had planned on mounting a camera on a geared up servo so that the camera could rotate independently of the wall following behavior. However, after building this assembly we decided to abandon the idea in favor of mounting the camera much lower. The lower vantage point gave us better visibility of the ball and facilitated alignment better. We also anticipated that trying to trouble shoot the alignment of the geared up servo (it wouldn’t stop at the same place each time) was going to consume too much time.

Contest Evaluation

It only took us about 15 seconds after seeing the course to realize that our robot would have difficulties. We had fully tested our robots ability to track and capture balls as well as align and shoot for goals; however, the staff decided to put two goals next to each other, a condition we hadn’t coded for. Our method for checking the distance to the goal relied on a “yellow” area calculation taken from a processed image. The two adjacent goals provided twice the expected area and therefore caused our robot to think that he was twice as close as he really was. Before shooting our robot double checked the image and during this step was able to catch itself; however, the close proximity of the walls in that portion of the course caused the robot to reverse direction after turning away from the goal. This meant that once the robot realized it wasn’t in scoring range it would turn right and then the front IR sensor would trip and tell it to turn left, at which point it would once again see the goal. This meant that our robot was never able to recover from the double yellow area problem.

Although our robot wasn’t able to score any goals during the contest round it did capture all of the balls on the side of the map that it started for a total of 5 possesion points. Unlike many robots which scored possession points by accidentally catching balls with their wheels our robot skillfully captured each ball. One piece of advice that future teams might consider if they are thinking about using a conveyor belt system is that if you make the front roller extend out from the body of the robot about an eighth of an inch the robot will actually be able to capture balls that are directly adjacent to walls (provided the ball isn’t in a corner); however, you must be careful that your conveyor is designed such that if the robot confronts a wall head on the belt doesn’t bind and break the belt drive motor.

We were mechanically pleased with our robots performance, all of our deliberate design choices functioned as expected. The software short comings that lead to our defeat where not the product of oversight or inability; in fact, the night of impounding Marcos actually debated trying to implement a multiple goal handling case but due to a lack of time and sleep we weren’t able to implement it.