Jump to content
Sign in to follow this  
Guest abulaafia

Strange FSX file accesses....

Recommended Posts

>This depends on the highest QMID level authored in the>scenery, as I recall. Uh, WHAT depends on the highest QMID level and WHAT scenery? Sorry, Phil. Without some contectual references I can't make any sense from this. And even reading the replied to post doesn't help alot in terms of what I'm attempting to learn. This is with respect to your previious reply that stated: "the terrain engine loads scenery that is within 4.5 tiles of the current location." I'd like to know what the size of the tiles are that are used by FSX to determine the '4.5 tile' loading criteria. Is this tile size a (cough) VARIABLE? I'm trying to understand the apparent aberant behavior I'm seeing and see if I can find some element in the addon scenery files I'm seeing loaded that might be contributing to this issue. If it's 'normal' for scenery files from 3000-7000 miles or more distant from the aircrafts currenty postion to be loaded, then I'll stop searching for problems and resort to kludging my scenery.cfg file for discrete flights. Paul


Wide-5.jpg

Share this post


Link to post
Share on other sites

IIUC all BGL files have headers containing lat and long bounds for the file. This is read when building the scenery database so that the sim knows what files to access. I believe it is known that MS can create object libraries that are geo-locked, but our libraries built with BGLComp don't do this. There also could be faults in BGLComp or 3rd party compilation tools that don't compute the bounds properly? scott s..

Share this post


Link to post
Share on other sites

>But the way>you put it is like telling someone they have no right to>criticize a movie unless they are a>director/producer/screenwriter all rolled into one, no right>to criticize a book unless they are an>author/editor/publisher. >> Are you talking to me? What did I "put"?RhettAMD 3700+ (@2585 mhz), eVGA 7800GT 256 (Guru3D 93.71), ASUS A8N-E, PC Power 510 SLI, 2gb Corsair XMS 3-3-3-8 (1T), WD 150 gig 10000rpm Raptor, WD 250gig 7200rpm SATA2, Seagate 120gb 5400 rpm external HD, CoolerMaster Praetorian


Rhett

7800X3D ♣ 32 GB G.Skill TridentZ  Gigabyte 4090  Crucial P5 Plus 2TB 

Share this post


Link to post
Share on other sites

That doesn't seem like something the program should be doing, for sure.I wonder what happens if you disable all scenery areas, except for the few over which you are actually flying? Then, I wonder if you notice any performance benefit? Because performance is the crux of the matter.RhettAMD 3700+ (@2585 mhz), eVGA 7800GT 256 (Guru3D 93.71), ASUS A8N-E, PC Power 510 SLI, 2gb Corsair XMS 3-3-3-8 (1T), WD 150 gig 10000rpm Raptor, WD 250gig 7200rpm SATA2, Seagate 120gb 5400 rpm external HD, CoolerMaster Praetorian


Rhett

7800X3D ♣ 32 GB G.Skill TridentZ  Gigabyte 4090  Crucial P5 Plus 2TB 

Share this post


Link to post
Share on other sites

perhaps you could please consider using a bit less attitude to start? that helps make one want to respond.do you have the FSX SDK installed?you can load up dem4km.bgl in TMFViewer and then overlay the QMID levels to get an idea of the extent of a QMID ltile at a specific level. for each level the tile is fixed in extent. that is why the extent of the range of "4.5 tiles" depends on the highest QMID level contained in the add-on scenery.at QMID level 5, the entire US is within 4.5 tiles of each other. See 185031.jpg attached.at QMID level 6, then yes there would be something out of sorts. See 185029.jpg attached.if you dont have the SDK installed, and are not familiar with it and authoring scenery, there is no way to explain what is happening. if you do have the SDK installed, you can then examine the scenery bgl using TMFViewer and "see" what level it is authored at and how many tiles at that level separate the sceneries in question. then and only then can you determine if this load pattern makes sense. a priori you cannot determine anything using filemon.another thing that can really hurt file access is the authoring pattern of the scenery. does it aggregrate the textures into larger ones so there are not a plethora of small textures? this is especially important if there is a lot of transparency in the scenery as that can also force repeated loads.in general, if you are using the default scenery and you do not see these sorts of access patterns, and you only see them when using 3rd party scenery that points to an issue with the 3rd party scenery authoring and not the engine.another instance of this dependence on the authoring of the 3rd party scenery is the difference between Horizon and TileProxy as I indicated in my last "Blurries" post on my blog. Horizon aggregates their source textures so their scenery generates on the order of 4k file accesses for a specific location as oppossed to 144k accesses for TileProxy in the same geo-location as discussed on the Horizon forums. That is ~35x ( almost 2 orders of magnitude ) the number of file accesses for 2 products that do the same thing. naturally that is something the engine has a hard time smoothing out.does that help clarify?

Share this post


Link to post
Share on other sites

Hey Phil-I have a comment that is totally unrelated to the current discussion. I would like to personally thank you and your crew for producing an outstanding product. I recently built my first PC which was quite an upgrade from my old Dell. With my current system, FSX runs perfect. I feel fortunate and grateful that I can turn on my computer and fly realistically pretty much anywhere in the world, thanks to you and your team. I also have learned a lot from your posts here at AVSIM and at your blog. I don't envy you having to deal with posters who may or may not know what they're doing but I guess that's why you make the big bucks :-) Good luck and keep up the great work!Markps. Will Acceleration install correctly on top of FXS/SP1/SP2? that's what I currently have and everything is running really well.

Share this post


Link to post
Share on other sites

To Phil Taylor:Thanks for FSX and thanks for your unlimited patience to deal with users whose strength of judgement is reverse proportional to their knowledge ..To Paul (original poster):The fact that (except for Reno) only addon scenery leaves these big footprints in filemon is a strong hint that we have a data problem, not a code problem.Please look at the following docs in the FSX SDK:- Environment Kit / Terrain SDK / Terrain and Scenery- Environment Kit / BGL Compiler SDK / Compiling BGLThe world scenery is divided into 8x12=96 rectangular tiles (at QMID level 4). These are the four digit named subdirectories in the scenery folder. Each of these tiles is further subdivided into 8x8=64 subtiles (at QMID level 7). Most bgl files stick to this tiling scheme.Here is a somewhat simplified explanation of whats going on:FSX scenery engine needs to know when to load a bgl file. So it looks at the bgl file header to find out what area of the world that bgl file corresponds to. FSX uses an index of all scenery files to look up those areas. If the aircraft is inside or close to (4.5 tiles) that area, FSX loads the bgl file, otherwise it doesnt.It is the scenery authors task to minimize that area. Lets assume the author made a typo and defines a VOR range of 5000 nm (instead of 50 nm). Now draw a circle of 5000 nm around that VOR, then draw the smallest rectangle that contains the circle. If that rectangle overlaps the 4.5 tile neighbourhood of the aircraft, FSX has to load the bogus bgl, since the bgl wants to be loaded ...I recommend that you contact the scenery authors and ask them to fix their scenery. Maybe they have a typo in their sources, maybe they did not use the FSX SDK tools but earlier or other tools ..Martin

Share this post


Link to post
Share on other sites

>For the next FS'xx" it might be nice to have selectable>'Addon Scenery' directories that can be enableddisabled similar to>having the 'Simobjects' entries in the fsx>CFG file.>That way one could group addon scenery by regions in an>appropriately named folder...'Addon_Scenery Western NY' or>some such and be able to enabledisable that folder and all>the sub-folders from the menu while still being able to have>seperate folders within that folder containing scenery, etc for>discrete areas without having to mix alot of files together in one>'scenery' and 'texture' folder under the main folder.Why wait, Paul? Who said you can't have as many sub-folders as you wish?We've been able to do that since FS9 at least! I don't have any addon scenery for FSX yet, so I'll have to use my FS9 "Full Version" to illustrate......but in FSX you don't even have to exit the sim and restart it! ;)Check out my FS9 Addon Scenery folder! I can go to the Scenery Library and enable/disable any of the installed scenery areas as I wish. I could have a virtually unlimited number of such sub-folders defined. Each one has it's own ..scenery and ..texture sub-folders.Directory of K:FS9Addon Scenery12/20/2007 12:56 PM


Fr. Bill    

AOPA Member: 07141481 AARP Member: 3209010556


     Avsim Board of Directors | Avsim Forums Moderator

Share this post


Link to post
Share on other sites

>>For the next FS'xx" it might be nice to have selectable>>'Addon Scenery' directories that can be enableddisabled>similar to>>having the 'Simobjects' entries in the fsx>CFG file.-SNIP->>Why wait, Paul? Who said you can't have as many sub-folders as>you wish?>>We've been able to do that since FS9 at least! I don't have>any addon scenery for FSX yet, so I'll have to use my FS9>"Full Version" to illustrate...-SNIP-Perhaps I didn't adequately describe what I meant here, Bill.I have 60 folders in my "Addon Scenery 2" folder on my D: driveand another 27 in the default FSX "Addon Scenery" folder.The thing is one must enable or disable each subfolder individually.What I was trying to describe was a method to group, say, all thesubfolders for a particular region or era into a unique folderand be able to enabledisable THAT folder, thus enablingdisabling all the subfolders within.For instance, in the case of SimObjects, I have an "Aircraft"folder that contains all my FS9 imports, an "AI Aircraft" folder for all my non-military AI, and an "AI


Wide-5.jpg

Share this post


Link to post
Share on other sites

-SNIP->To Paul (original poster):>>The fact that (except for Reno) only addon scenery leaves>these big footprints in filemon is a strong hint that we have>a data problem, not a code problem.>Thanks for your concise explanations below.Well Martin, I do not discount that there MAY be a 'code problem' but I'm also concerned that it may be causing some aspects of FSXto not perform as anticipated. That's what I'm trying to get a handle on.Also, I'm not entirely certain that the 'out of area' BGL loading is restricted to addon scenery. It's just that these elements are eaasier to spot since thay have file names that are decriptive in nature. I use filtering on FileMon to exclude all the other FSX accessesto SceneryWorld....whatever. There are ceratainly alot of those.>Please look at the following docs in the FSX SDK:>- Environment Kit / Terrain SDK / Terrain and Scenery>- Environment Kit / BGL Compiler SDK / Compiling BGL>I've read through the SDK in various areas whenever I'm doing any tweaking. The scenery and terrain naration isnot the most easily understood of documents. >The world scenery is divided into 8x12=96 rectangular tiles>(at QMID level 4). These are the four digit named>subdirectories in the scenery folder. Each of these tiles is>further subdivided into 8x8=64 subtiles (at QMID level 7).>Most bgl files stick to this tiling scheme.>So, in the discussions here, can one safely assume that"tile" refers to the QMID level 4 size?If so, then 4.5 tiles is one EXTREMELY LARGE area and it wouldbehove me to ask just WHY the scenery engine would need to load an area that large?Even at QMID level 7, your're talking 1/4 of the USA.>Here is a somewhat simplified explanation of whats going on:>>FSX scenery engine needs to know when to load a bgl file. So>it looks at the bgl file header to find out what area of the>world that bgl file corresponds to. FSX uses an index of all>scenery files to look up those areas. If the aircraft is>inside or close to (4.5 tiles) that area, FSX loads the bgl>file, otherwise it doesnt.>Again, the magic '4.5 tiles'...without a definative value forwhat constitues a 'tile', the statement is rather meaningless.>It is the scenery authors task to minimize that area. >>Lets assume the author made a typo and defines a VOR range of>5000 nm (instead of 50 nm). Now draw a circle of 5000 nm>around that VOR, then draw the smallest rectangle that>contains the circle. If that rectangle overlaps the 4.5 tile>neighbourhood of the aircraft, FSX has to load the bogus bgl,>since the bgl wants to be loaded ...>Yes, I understand all of that and although it's possible that the Khrabrovo scenery has a glitch, some testing I did earlier tonight tended to point to some other issues related to howmany files there are from any scenery being loaded and the size of the files.For instance, I enabled alot of my addon scenery and mesh andstarted a flight sitting on the ramp a KROC. I saw inital loading ofHawaii Mesh, West Coast mesh, KSEA photo scenery, and a whole lot of others. However, there was no additional loading of thoseelements as I sat on the ramp or flew a circuit around KROC.I enabled Khrabrovo with only the AFAD files installed and again,the same activity as above.I then added in 1/2 of the 144 BGL files in the Khrabrovo scenery.Again. I saw the earlier far off elements loaded as well as the files I had just enabled. Again, no subsequent loading ofthe addon scenery files I was tracking.I then added a few more of the Khrabrovo BGL's and then started seeingthe periodic updates.Now here is another "what the hey"...I have TrackIR and as I'msitting on the ramp, static, and looking straight ahead, I see nowfile activity but as soon as I turn my head left or right I see a burstof activity as all the addon scenery files get reloaded!I tried various combinations of file mixes from the Khrabrovo scenery BGL's but there was no decernable pattern other than the total size of theBGL's enabled, or so it appeared.I saw similar 'move head...get out of area BGL's to reload' activitywith Khrabrovo disabled and some additional addon scenery enabled.I was seeing the KSEA "KSEA_asbuilt_v2_photo.bgl" being loadedabout every second as I sat motionless on the ramp a Rochester.That file appears to straddle only 3 tiles at QMID level 11.IMO this is clearly a 'problem' for it to be loaded when I'msitting on the east coast.This is rather interesting but doesn't relate as to WHY thosefiles are being loaded in the first place. That is what I'm strivingto understand here.>I recommend that you contact the scenery authors and ask them>to fix their scenery. Maybe they have a typo in their>sources, maybe they did not use the FSX SDK tools but earlier>or other tools ..>>>Martin>I'm inclined to think that it's not Khrabrovo but rather the quantity andor size of various files that are being loaded and require reloading, for whatever reason. Paul


Wide-5.jpg

Share this post


Link to post
Share on other sites
Guest JeanLuc_

Hi,I've been reading the discussion and all of a sudden a question spakled my interest: since the granular size of a tile is variable, based on aithoring defined parameters, why is FS loading a number of tile per-tile number, instead of per-nautical mile distance?I mean, if 1 tile is 1/4 of the USA, why loading 4.5x4.5 of them, instead of just 2x2? (a 2x2 square and you are located withing the centers of those 2x2 tiles?And if 1 tile size is smaller than the distance to visible horizon and/or longest nav receiver distance capability, then loading YxY tiles up to cover just that maximum necessary distance?

Share this post


Link to post
Share on other sites

>Hi,>>I've been reading the discussion and all of a sudden a>question spakled my interest: since the granular size of a>tile is variable, based on aithoring defined parameters, why>is FS loading a number of tile per-tile number, instead of>per-nautical mile distance?>-SNIP- Hi JeanLuc, That question has crossed my mind as well. It seems to me that a metric that is more closely tied to the actual flight conditions would be a better choice than some 'invisable' variable. If I'm flying an aircraft with fuel onboard for a max range of 1000 miles and max cruise speed of 230 knots, one should be able to determine how far out one needs to load scenery elements based on time vs distance. In the cases I'm seeing, it is not only things like mesh or landclass being loaded but individual scenery items...buses, tires, papi, etc. And the mysterious part is that the associated textures are never accessed. Paul


Wide-5.jpg

Share this post


Link to post
Share on other sites

Hi Paul,>Also, I'm not entirely certain that the 'out of area' BGL loading is restricted to addon scenery. It depends on the bgl files content, e.g. the six bgl files containing geopolitical boundaries each cover 90 degrees latitude and 120 degrees longitude (QMID 2). On the other hand, APX*.bgl files (which contain airports) cover an area of only 3x3 degrees (QMID 7). ATX*.bgl contain navigation aids information and cover an area of 22x22 degrees (QMID 4). As far as I understand, each bgl file can have its own QMID level setting in the header, even wrong ones .. ;-P>So, in the discussions here, can one safely assume that "tile" refers to the QMID level 4 size?Nope, see above. However I think Phil referred to QMID 7, but i am just guessing ..It doesnt make sense to load all airport buildings of KSFO when youre sitting at KJFK. However it makes sense to load the waypoints / intersections and victor / jet airways between KJFK and KSFO ..The internal structure of each bgl file is organized like an directory. So if FSX accesses a bgl file, it might load it as a whole or just some small subset of interesting parts, e.g. when sitting at KJFK, FSK might load the bgl containing KJFK as a whole (to get all runways, taxi ways, parking spots, buildings, and whatever ...). When FSX loads the bgl file with KSFO (while still sitting at KJFK), it might extract only the ARP coordinates of KSFO and maybe the approach and transition info, and leave the rest for later when you get closer. Filemon doesnt tell you what exactly FSX reads from those files, just that FSX opened it.The standard scenery is about 12 GB, so FSX simply cannot keep it all in memory, but has to load and reload scenery info again and again to limit its own memory footprint. FSX has no other choice than to rely on the bgl header info being correct. If a bgl file says it covers area X, FSX has to load it when in area X or close to it.Now one could argue why 4.5 tiles instead of 2 or 6 ..I have no clue why 4.5 is the magic number, but im sure if MS ACES could use a smaller value they would use it, just to minimize FSX memory footprint.There are a number of different factors that determine that magic number:- how much scenery is visible to the eye from max altitude ?- what nav info is "visible" to the gps at max range ?>This is rather interesting but doesn't relate as to WHY those>files are being loaded in the first place. That is what I'm>striving to understand here.To understand what FSX is doing (and WHY !), I would recommend to turn of all addon scenery (assuming that the standard scenery files always use the smallest needed QMID), then fly from x to y and check whats happening in filemon, then look at TmfViewer to check if that makes some sense. After a while you will get a "feeling" for necessary vs. unnecessary loads.After that, you can activate addon sceneries one by one and repeat the same flight again. If something way outside the loaded standard scenery is touched, it is a very strong hint that the QMID header info in that addon scenery is to high.The latest version of TileProxy has a nice map overlay for monitoring the file access, however i dont know how to use that map thing without TileProxy. But it might help to get a "feeling" for the loading patterns ..Martin

Share this post


Link to post
Share on other sites

> In the cases I'm seeing, it is not only things like mesh or > landclass being loaded but individual scenery items...buses, > tires, >papi, etc.MS Aces uses certain conventions what they put in what bgl files (e.g. airports in APX at QMID 7, navaids in ATX at QMID 4, vector data in CVX at QMID 7, ...). I am sure these conventions are fine tuned to the FSX engine for optimal performance and minimal memory footprint.However the BGL compiler does not enforce these conventions. A somewhat clueless scenery author can throw all kind of unrelated stuff in the same bgl file. A smart scsnery author would analyze the standard scenery to learn the conventions and follow them.And then: what was ok in FS9 might be bad in FSX ..So just recompiling unchanged sources of an FS9 scenery with FSX SDK will give a working (error free) FSX scenery, but at bad performance.> And the mysterious part is that the associated textures are> never accessed.Well, textures are only needed when the thing becomes visible to the eye. Position info might be needed much earlier, e.g. when the GPS can see it ..Martin

Share this post


Link to post
Share on other sites

The old FSManager utility had the concept of scenery sets. You could assign scenery areas to sets and turn them on/off as a group. something like that would be useful.scott s..

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Tom Allensworth,
    Founder of AVSIM Online


  • Flight Simulation's Premier Resource!

    AVSIM is a free service to the flight simulation community. AVSIM is staffed completely by volunteers and all funds donated to AVSIM go directly back to supporting the community. Your donation here helps to pay our bandwidth costs, emergency funding, and other general costs that crop up from time to time. Thank you for your support!

    Click here for more information and to see all donations year to date.
×
×
  • Create New...