A Performance Comparison of Multi-Hop Ad Hoc Network Routing Protocols

This paper simulates four MANET routing protocols with a somewhat more realistic model than previous work, and evaluates how they perform under different degrees of node mobility. The paper includes a detailed description of each protocol, which was great to have. The high-level conclusion is that DSR and perhaps AODV are best in terms of delivery success rates and overhead (although this is not stated in the intro; most of the focus is on methodology).

Although this paper was very detailed, there were several things that really bothered me about the methodology and the evaluation:

• The physical later model is still very simplistic. It's really hard to match the real world with simulations in something as complicated as wireless.
• The paper modified the algorithms studied in a variety of ways. While this could be seen as a good thing because it makes it more fair to each algorithm, there is the question of when to stop. The improvements in the paper clearly had a positive effect on the algorithms. How do we know whether some equally simple improvements won't drastically improve the algorithms that performed badly?
• The constants for the algorithms (timeouts, refresh intervals, etc) seem to be pulled out of a hat.
• The movement and communication models are very arbitrary. For example, few groups of things really move along random waypoints, or do nothing for the first 600 seconds and then all start moving at once. The 1500x300 (5x1) field is just really bizarre. So is having ony CBR sources. Just running a lot of experiments with the random waypoint model isn't enough, especially if you're only going to present averages of performance in the experiments, and not distributions. The paper even says that performance is very sensitive to movement, which should be a sign that it's important to validate a model, maybe base it on traces of real use cases, and try a variety of scenarios which are not just random samples from the same model.
• Standard deviations aren't shown in the results, and it's hard to tell why certain protocols do badly. Maybe the algorithms were designed specifically to deal with particular scenarios - why then simulate them all in the same scenario?

Overall I would feel better if a paper like this had a strong conclusion and provided insight into why things behave the way they do in real life, rather than creating a scenario and seeing how they behave there.
In terms of course organization, this paper should come before the path metric one, since it describes the DSDV and DSR algorithms used in the path metric paper.