第二次运行托管应用程序显示的性能与第一次不同-Runningamanagedapplication2ndtimeshowsdifferentperformancethan1st

作者：手机用户2502914467 | 来源：互联网 | 2023-09-02 18:54

IhaveabenchmarkingapplicationtotestouttheperformanceofsomeAPIsthatIhavewritten.Int

I have a benchmarking application to test out the performance of some APIs that I have written. In this benchmarking application, I am basically using the QueryPerformanceCounter and getting the timing by dividing the difference of QPC values after and before calling into the API, by the frequency. But the benchmarking results seems to vary If I run the application (same executable running on the same set of Dlls) from different drives. Also, on a particular drive, running the application for the 1st time, closing the application and re-running it again produces different benchmarking results. Can anyone explain this behavior? Am I missing something here?

我有一个基准测试应用程序来测试我编写的一些API的性能。在这个基准测试应用程序中,我基本上使用QueryPerformanceCounter并通过在频率调用API之前和之后除以QPC值的差异来获得时序。但基准测试结果似乎有所不同如果我从不同的驱动器运行应用程序(在同一组Dll上运行相同的可执行文件)。此外,在特定驱动器上,第一次运行应用程序,关闭应用程序并再次重新运行它会产生不同的基准测试结果。谁能解释这种行为?我在这里错过了什么吗?

Some more useful information:

一些更有用的信息:

The behavior is like this: Run the application, close it and rerun it again, the benchmarking results seems to improve on the 2nd run and thereafter remains same. This behavior more prominent in case of running from C drive. I would also like to mention that my benchmark app has an option to rerun/retest a particular API without having to close the app. I do understand that there is jitting involved, but what I dont understand is that on the 1st run of app, when u rerun an API multiple times without closing the app, the performance stabilizes after a couple of runs, then when you close and rerun the same test, the performance seems to improve.

行为是这样的:运行应用程序,关闭它并再次重新运行,基准测试结果似乎在第二次运行时得到改善,之后保持相同。在从C盘运行的情况下,此行为更加突出。我还想提一下,我的基准测试应用程序可以选择重新运行/重新测试特定的API,而无需关闭应用程序。我知道有涉及jitting,但我不明白的是,在第一次运行应用程序时,当你多次重新运行API而不关闭应用程序时,性能在几次运行后稳定,然后当你关闭并重新运行时同样的测试,性能似乎有所提升。

Also, how do you account for the performance change when run from different drives?

另外,如何在从不同驱动器运行时考虑性能变化?

[INFORMATION UPDATE]

I did an ngen and now the performance difference between the different runs from same location is gone. i.e. If I open the benchmark app, run it once, close it and rerun it from same location, it shows same values.

我做了一个ngen,现在来自同一位置的不同运行之间的性能差异消失了。即如果我打开基准测试应用程序,运行一次,关闭它并从同一位置重新运行它,它显示相同的值。

But I have encountered another problem now. When I launch the app from D drive and run it a couple of times (couple of iterations of APIs within the same launch of benchmark prog), and then from the 3rd iteration onwards, the performance of all APIs seems to fall by around 20%. Then If you close and relaunch the app and run it, for first 2 iterations, it gives correct values (same value as that obtained from C), then again performance falls beyond that. This behavior is not seen when run from C drive. From C drive, no matter how many runs you take, it is pretty consistent.

但我现在遇到了另一个问题。当我从D驱动器启动应用程序并运行它几次(在同一个基准测试版本中的几次API迭代),然后从第3次迭代开始,所有API的性能似乎下降了大约20% 。然后,如果您关闭并重新启动应用程序并运行它,前2次迭代,它会给出正确的值(与从C获得的值相同),然后性能再次超出该值。从C驱动器运行时看不到此行为。从C盘开始,无论您运行多少次,都非常一致。

I am using large double arrays to test my API performance. I was worried that the GC would kick in inbetween the tests so I am calling GC.Collect() & GC.WaitForPendingFinalizers() explictly before and after each test. So I dont think it has anything to do with GC.

我使用大型双数组来测试我的API性能。我担心GC会在测试之间启动,所以我在每次测试之前和之后明确地调用GC.Collect()和GC.WaitForPendingFinalizers()。所以我不认为它与GC有任何关系。

I tried using AQ time to know whats happening from 3rd iteration onwards, but funny thing is that When I run the application with AQ time profiling it, the performance does not fall at all.

我尝试使用AQ时间来了解从第3次迭代开始发生的事情,但有趣的是,当我使用AQ时间分析它运行应用程序时,性能根本没有下降。

The performance counter as does not suggest any funny IO activity.

性能计数器并不表示任何有趣的IO活动。

Thanks Niranjan

4 个解决方案

#1

I think there are a combination of effects here:

我认为这里有一系列效果:

Firstly, running the same function within the test harness multiple times, with the same data each time, will likely improve because:

首先,多次在测试工具中运行相同的功能,每次使用相同的数据,可能会改进,因为:

JIT compilation will optimise the code that is run most frequently to improve performance (as mentioned already by Cory Foy)

JIT编译将优化最常运行的代码以提高性能(如Cory Foy所述)

The program code will be in the disk cache (as mentioned already by Crashwork)

程序代码将在磁盘缓存中(如Crashwork所述)

Some program code will be in the CPU cache if it is small enough and executed frequently enough

如果程序代码足够小并且执行得足够频繁,那么它们将位于CPU缓存中

If the data is different for each run of the function within the test harness, this could explain why closing and running the test harness again improves results: the data will now also be in the disk cache, where it wasn't the first time.

如果测试工具中每次运行该功能的数据不同,这可以解释为什么再次关闭和运行测试工具可以改善结果:数据现在也将位于磁盘缓存中,而不是第一次。

And finally, yes, even if two 'drives' are on the same physical disk, they will have different performance: data can be read faster from the outside of the disk platter than the inside. If they are different physical disks, then the performance difference would seem quite likely. Also, one disk may be more fragmented than the other, causing longer seek times and slower data transfer rates.

最后,是的,即使两个“驱动器”位于同一物理磁盘上,它们也会有不同的性能:数据可以从磁盘盘片的外部读取得比内部更快。如果它们是不同的物理磁盘,那么性能差异似乎很可能。此外,一个磁盘可能比另一个磁盘碎片更多,导致更长的搜索时间和更慢的数据传输速率。

#2

Running an application brings its executable and other files from the hard drive into the OS's disk cache (in RAM). If it is run again soon afterwards, many of these files are likely to still be in cache. RAM is much faster than disk.

运行应用程序将其可执行文件和其他文件从硬盘驱动器带入操作系统的磁盘缓存(在RAM中)。如果它很快再次运行,其中许多文件可能仍然在缓存中。 RAM比磁盘快得多。

And of course one disk may be faster than another.

当然,一个磁盘可能比另一个磁盘更快。

#3

Yes. It's called Just-In-Time compiling. Basically your app is deployed as MSIL (the Microsoft Intermediate Language) and the first time it is run it gets converted to native code.

是。它被称为Just-In-Time编译。基本上,您的应用程序部署为MSIL(Microsoft中间语言),并且第一次运行时它将转换为本机代码。

You can always run NGen (see the above article), or have a warm up period in your performance testing scripts where it runs through the scenario a couple of times before actually benchmarking performance.

您可以随时运行NGen(请参阅上面的文章),或者在性能测试脚本中预热一段时间,在实际基准性能测试之前,它会在场景中运行几次。