Originally published on https://www.justflight.com/articles/developers-diary-traffic-global-for-xplane-11

Welcome back! After dealing with providing aircraft models and the actual traffic database – at least partly – last month, it’s time to take a step back and look at some of the design choices that were made, and why.

The big question of course is “what are we trying to achieve?”. Well… traffic. In the sim. Speaking as a real-life (if very occasional) pilot who flies near London I’d love nothing more than to remove a great deal of traffic! However, there’s no question that in-sim there are far fewer other aircraft around and if you’re going for “as real as it gets”, that’s a problem.

So, once again, what are we aiming for? You might say “as real as it gets but c’mon guys it’s a simulator”. I was reading a forum thread the other night about a completely different traffic product which had a long sequence of posts along the lines of “… yeah, but my local airport, Hicksville Muni, added three parking spots on the grass last week and does this product that was released 18 months ago use them? Does it heck. What a waste of money.”

One of the earliest design choices to be made was how dynamic the traffic would be. In other words, it would be technically possible to use a live traffic feed, or some kind of subscription service with continual, drip-fed updates, to keep things absolutely bang up to date – but that’s a lot of extra work, infrastructure, maintenance, testing, data licensing fees and so on. Forever. There are other downsides too; you need a permanently-on good quality internet connection, not too tricky nowadays but still a consideration. Differences between real-world and simulated airport facilities, which are always going to exist, will be even more obvious.

On the other hand, a static database of flights, like Prepar3D uses and TGXP will use, is simple to update periodically (by the developers) and can be changed at will (by the end users) if they really want to. On the plus side, it’s a single unchanging file, so it’s simple. (This is absolutely not true – see below!). On the downside, it’s static. If an airline changes a schedule in the real world then the file doesn’t magically update to match. Another drawback is that the file format that’s being used – at least right now – is identical to the one used by Prepar3D, to let us re-use the existing files and management tools, and this format doesn’t allow for start and end dates. That means that if a particular flight runs only from March to September, there’s no way of dealing with that.

That’s a shame, and I’d love to get this feature in. It might – that’s might, as in maybe, maybe not – make it in yet. I’m in control of the traffic database format so it would be very easy to add. It also means more complex preparation of the original data, writing a new and more complex traffic database compiler, and no longer being able to share either the compiled database or the front-end software with the Prepar3D version therefore having to completely re-write that. That’s all time that could be spent making planes do things like fly round corners, or remember to lower their landing gear on approach.

So will you be able to stand in the observation gallery of a real-world airport with X-Plane running on your laptop and watch both a physical aircraft and it’s simulated partner taking off at the same time? Actually, yes, in some cases, but – and here’s the thing – that’s not the point. Will the traffic in-sim be as busy on a typical Tuesday as it would for the real airport? Should be, more or less. Depends. Will it feel different on Sundays? Again, should be. We’re aiming to make a given airport feel as busy as it ought to, on a given day of the week and time of day, where possible using real-world flight data from the recent past. If you’re planning on being able to launch X-Plane to check if your wife’s flight landed on time and should you start getting some food ready for when she gets in, there’s probably more appropriate tools out there!

It looks like this diary is turning into “The Ugly Truth About Traffic”, so let’s keep going and then never have to touch it again!

One of the other comments in that forum thread went along the lines of: “Surely you just download some commercial data, reformat it the way the simulator wants, and ship it? A few hours’ work at most. How hard can it be?” Well, that’s genuinely a very good question, so let’s find out. (Spoiler warning: you might want to get a stiff drink at this point!)

First up, the real world usually differs from the simulated one. More specifically, airports can come and go, or change their four-letter ICAO identification codes, meaning that the downloaded flight data might not match the world that the simulator knows about. Even more specifically than that, parking availability is often different. Very different. If a real-world airport can handle 50 planes at once while it’s simulated version can only handle 40, that’s potentially 10 flights at any given moment that can’t happen, and that could be hundreds over the course of a single day. Those flights would all at some point fly to and from other airports, so even if those airports can handle the number of arriving planes, they still won’t be arriving.

It’s not just how much parking that’s the issue here, it’s also the type of parking. Different types of aircraft need different grades of parking slot, so if the simulated airport has the full 50 slots but they’re graded too small, that’s another difference that needs to be worked out. Usually simulated airports are short of parking compared to their real-world equivalents. Many airlines also pay for reserved parking at given airports, excluding all other airlines, so this also needs to be checked. Some parking slots are so close to others that only one of them can be used at a time; sometimes they’re called something like “257L” and “257R”, but sometimes not so it’s not a reliable way of checking.

What else needs to match? Well, the next most obvious thing is the aircraft. Each commercial flight record will have an aircraft type, so we need to make sure that that type is available in the simulator, with the correct livery for the right airline. That means it needs to be cross-referenced with a list of every airline in the world and every commercially-used aircraft in the world, and each combination of these cross-referenced with the types that are provided with the traffic package, to see which routes we can use. We might also want to look for alternatives; if there’s no specific livery for a Garuda B737-8, what’s the nearest equivalent that we do have? Oh, and remember to check that the replacement aircraft type can fit into the same parking slot size! And what if we don’t have anything for Garuda? What’s the most appropriate airline to use instead? Or should we just use unpainted aircraft? Or maybe drop the flights?

Getting a little trickier now, right? Take a deep breath and read on!

Next up are schedule changes. The commercial flight database will cover a period of time, and during that period, flights might change. So, you could end up with flight AB1234 leaving an airport at 10:05 on a Tuesday and 10:15 on the same Tuesday if you’re not paying close attention to dates. It might also change routes, or aircraft types, or day of the week, or any combination of these, so we need to look out for these changes and decide which version of the flight to use. “All of them, just in case” isn’t really a useful option because that takes up extra parking slots, which we already know are in high demand.

Finally, we have the small problem that the commercial flight database is a list of flights, while what we need are routes. Prepar3D and TGXP both work on the principle that planes shouldn’t ever just pop into existence at a gate and then pop out of existence again after landing. It makes sense if you think about it; how many airlines buy a new plane, load it with passengers and fly it to the destination, then scrap it? If you have a plane that flies from A to B, at some point you want it back at A again ready to run the same flight the next day. This means that the simulator expects to be given a list of routes, where a plane leaves airport A, goes some other places, and ends up back at A again. It also wants to be told how often the route starts, ranging from two hours up to eight weeks. And don’t forget that the plane needs a parking slot for all the time it’s not in the air after the route finishes, while it’s waiting for it to start again.

So, where are we now? We have a list of around 200,000 flights. Some are almost-but-not-quite duplicates, so these need to be found and we need to decide which version to use. Some fly to airports that might have different IDs, or just not exist at all, in the simulator. The flight times, which are all in local time, need to be converted to UTC time which means getting accurate timezone and daylight savings data for every airport that’s used. Some flights will use aircraft, airlines or combinations of them that don’t exist in the simulator and need to be mapped onto something else. We need to work out which of these flights represent a single aircraft landing at multiple airports before it reaches it’s final destination, and split these multi-hop flights down into individual stages. Most importantly, we need to work out where each plane goes after it has landed, guessing at which flight out of the destination airport represents that plane getting back to it’s starting point. It might be going via somewhere else – or several somewhere elses – first, and there might have been more than one version of any stage of the flight. We might need to take several stabs at this if another plane ends up “stranded” with no outgoing flight. Since the sim’s airports and parking are usually different to real ones, some flights won’t fit into the airports when they land, so we need to work out what parking is available in the sim, and of what size, at each one of the sim’s 40-odd thousand airports, and track what planes of what sizes want to park there at all times regardless of whether they’re flying on a two-hourly cycle or a two-monthly one, remembering to check to see if using one parking slot blocks others nearby. Then we need to selectively drop flights that won’t fit, and just to be nice, see if any that used to not fit now do fit, because one of the dropped flights is now not landing at the airport that originally blocked the other blocked flight.

I genuinely don’t know if code-share flights are in there too. I haven’t dared look. And then there’s the question of each end-user having add-on airports with different parking availability.

So, to return to the original question: Can preparing the traffic database possibly be any more complicated than hitting “Save As…” in Excel? Well, yes. Yes, it can.

1… and getting this lot together…. well, just don’t go there.

Next month, you’ll be relieved to hear, I’ll probably be covering something far, far simpler – how to get all these flights into the sim without slowing it down.