Building a Better (Linux) GPU Benchmark
by Kristopher Kubicki on September 24, 2004 12:05 AM EST- Posted in
- Linux
Introduction
For those who follow our Linux reviews, we have made a lot of headway in the last two months. Our benchmarking has improved, our graph engine is top notch and we are working closely with all the major manufacturers to bring a definitive resource for Linux hardware to our readers. Today, we want to introduce everyone to our GPU Linux benchmarks and how we will run them in the future. This isn't a comparative analysis yet, but we won't keep you waiting long for that.The adherent flaw with any benchmark is that you, the reader, only receives a sampling of data - engineers and statisticians alike call this "data compression". When we sample data from a timedemo and format it into an average frames per second, we lose all sort of valuable data, such as what the lowest frames per second was, what the highest was, when the largest dip in FPS had occured, what the image looked like, and the list goes on. There have been a few attempts to convey more than just an average FPS in video benchmarks, most notably with FRAPS. FRAPS does not entirely address the issue of reproducibility and FRAPS runs on Windows only.
Fortunately, we have been graced with some very talented programmers who worked with us to build a benchmarking utility similar to FRAPS (on Linux) that we may eventually port over to Windows as well. Consider this to be our experiment in advancing our benchmarking methods while using Linux as our guinea pig. Eventually, we anticipate releasing the benchmark complete with source to the public.
Here is how our utility works, as explained by the lead developer, Wiktor Kopec.
"The program computes frames per second for an application that uses OpenGL or SDL. It also takes screenshots periodically, and creates an overlay to display the current FPS/time.You can read more about SDL and OpenGL. SDL is a "newer" library bundled with most recent Linux games (Medal of Honor: AA, Unreal Tournament 2004). In many ways, SDL behaves very similarly to DirectX for Linux, but utilizes OpenGL for 3D acceleration.
"This is accomplished by defining a custom SwapBuffers function. For executables that are linked to GL at compile time, the LD_PRELOAD environment variable is used to invoke the custom SwapBuffers function. For executables that use run-time linking - which seems to be the case for most games - a copy of the binary is made, and all references to libGL and the original glXSwapBuffers function are replaced by references to our library and the custom SwapBuffers function. A similar procedure is done for SDL. We can then do all calculations on the frame buffer or simply dump the frame at will."
Facebook
Twitter
RSS
23 Comments
View All Comments
- Saturday, October 24, 2009 - link
http://www.goph3r.com/mh">http://www.goph3r.com/mh(air jordan, air max, shox tn, rift, puma, dunk sb, adidas) nike jordan shoes 1-24 $32
lv, coach, chane bag $35
COOGI(jeans, tshirts, hoody, jacket) $30
christian audigier(jeans, tshirts, hoody) $13
edhardy(shoes, tshirts, jeans, caps, watche, handbag) $25
Armani(jeans, tshirts,) $24
AF(jeans, coat, hoody, sweater, tshirts)Abercrombie & Fitch $31
http://www.goph3r.com/mh">http://www.goph3r.com/mh
quanta - Friday, October 1, 2004 - link
If you are going to compare image quality, what is the basis of comparison? That means that a reference (software) renderer is needed. But even then, choosing right technique means ones need to draw extremely detailed scenes similar to CAD programs, but such renditions will be too slow. And of course, there's the issue of how to objectively define the 'right' way vs. 'wrong' way of optimization in ways that most can agree, and can be done by computers.In any case, we can't afford yet another GPU benchmark that is easily exploited (in bad ways, that is).
tygrus - Tuesday, September 28, 2004 - link
Nice start.I get frustrated by companies that delay support for <5% of the market because of small numbers when its the lack of support that is causing the low numbers. If ATI improved the drivers for linux then more people would use their existing ATI cards under linux and more importantly more people will buy ATI for their linux system instead of Nvidia. It's the issue of "what came first, the chicken or the egg", when their attitude should be more like "build it and they will come".
TrogdorJW - Tuesday, September 28, 2004 - link
Just a quick question here, as I see a potential performance issue with the "hacked" OpenGL/SDL framerate utility. You mention that the software will automatically capture screen shots periodically. That ought to increase the demand on the HDD and potentially the rest of the system for brief instants when the screen captures are done. Hopefully there will be multiple benchmarking runs done, one with the screen captures and one without. Although, I suppose we have to see what sort of difference the screen captures actually make in performance first. :)raylpc - Saturday, September 25, 2004 - link
I agree with fic. I'm going to buy a new AMD64 system and I will get Nvidia just because their linux support is way superior than ATi's, although ATi does make better cards.aaime - Saturday, September 25, 2004 - link
I hope you can add some 2D benchmarks as well, since, for example, the 2D performance of the ATI drivers really suck! A 2D perf. benchmark should involve also use of Render, text antialiasing and such to be representative.Anyway, thank you, it's nice to see some attention to the Linux world from you :-)
fic - Friday, September 24, 2004 - link
Probably the big reason that only 4% of owners of ATI cards run linux is that their support sucks. This is a self fulfilling thing. ATI doesn't support linux therefore if you are going to run linux you don't use ATI. I image that a huge percentage of linux users use Nvidia because they have decent drivers.Brule - Friday, September 24, 2004 - link
Great article. That's why they call it computer "science". Looking forward to the results on a personal linux-using level as well.javalino - Friday, September 24, 2004 - link
Well. I hope this type of benchmark didnt effect the continuos windows benchmarks. As the ATI said this week, 4% of owers of ati cards run linux, so, not much people want to see linux benchmarks, unless they are better than windows ones.Illissius - Friday, September 24, 2004 - link
Amazing. I've been looking for a tool like this for Linux for ages. Gimme now! Gimme now!