PugetBench for After Effects

At Puget Systems, one of our primary goals is to provide fast and reliable workstations that are perfectly tailored to each customer’s unique workflow. As a part of this, we publish an ongoing series of hardware articles for After Effects – as well as numerous other software packages – in order to discover what hardware configurations are optimal for these specific applications.

However, while this ensures that we are selling the right hardware, it does not give our customers a great idea of how much faster a new workstation might be compared to their existing system. In addition, with the amount of hardware that is currently available, it is impossible for us to test everything on our own, so we want to provide tools for other hardware reviewers to perform real world benchmarking in professional applications like After Effects.

To address both of these issues, we are making our After Effects benchmark available for download so that anyone can can perform the same testing we do in our hardware articles. Even better, this benchmark is compatible with both Windows and Mac-based systems which allows for cross-platform performance comparisons.

Be sure to check out the full list of benchmarks we have available!

System Requirements

• Adobe After Effects version 16.x, 17.x, 18.x, 22.x, 23.x, 24.x
  • Unsupported After Effects builds due to bugs:
    • After Effects 24.1 due to a bug with Cinema 4D / Cineware that causes a “File is damaged 33:7” error.
    • After Effects 22.5 due to a bug that causes a crash when the “Expressions” composition is opened.
    • After Effects 22.6 due to a bug that causes a crash when the “Expressions” composition is opened and an additional bug that halts the benchmark when a render is started via the renderAsync() function in the API.
• 24GB of memory (at least 18GB of which is available for Ae to use)
• Compatible with both Windows 10/11 and MacOS
• After Effects/OS language must be set to English

Note that this benchmark is still in BETA. Plug-ins and customized preferences in particular may prevent the benchmark from running properly. If you run into any issues, we recommend resetting your preferences either by following Adobe’s guide or by using our Adobe CC Preference Backup/Restore/Clear utility (Windows only).

How to run the benchmark

1. Add the benchmark plugin to Creative Cloud from the Adobe Exchange
2. Follow the directions to install the plugin on your system
3. Launch After Effects
4. Configure the necessary After Effects settings:
   • Enable “Allow Scripts to Write Files and Access Network” in “Edit -> Preferences -> Scripts & Expressions”
   • In Ae 22.x and newer, Disable “Composition -> Preview -> Cache Frames When Idle”. You will need to make an empty composition before you can change this setting (“Composition -> New Composition”)
5. Start the plugin from “Window->Extensions->PugetBench for After Effects”
6. Configure any settings you want to change (especially if you have a commercial license)
7. Run the benchmark!

When the benchmark is complete, it will give you an “Overall Score” and upload the results to to our online database (unless told not to with the commercial version). From the uploaded results, you can view the sub-scores and individual test results.

Troubleshooting

If you have any issues with the benchmark not properly running, first verify that your system meets the System Requirements and that the version of After Effects you are using supports it. If there are no issues with your system or software, there are a number of troubleshooting steps you can try:

• Make sure you are following each step listed in the How to run the benchmark section.
• Make sure you have at least 12GB allocated for After Effects to use (set in the memory section of preferences).
• Do not attempt to use the system while the benchmark is running. Defocusing the application window can break the benchmark in many situations.
• If you are on MacOS and have renamed your drive, the benchmark will not be able to get your system specs. The only work-around at the moment is to rename your drive back to “Macintosh HD”, or you can continue to run the benchmark without it being able to pull specs.
• Clear the application preferences and reset them to default using Adobe’s guide or our Adobe CC Preference Backup/Restore/Clear utility (Windows only).
• Disable Windows Defender or any real-time virus scanning programs when you run the benchmark.
• Ensure that your OS and After Effects is set to English and that the path to the benchmark script does not have any special or non-English characters.
• The benchmark does not work over an unmapped network drive (I.E. //192.168.1.2/Benchmark/). Mapped drives should work, but we highly recommend having the benchmark on a local drive as storage performance can affect the results.

Lower than expected performance can be caused by a number of factors, but the most common are:

• Outdated BIOS/drivers/software
• Running background processes (Antivirus, Windows Updates, etc.)
• Windows power profile (High Performance is recommended for the best results)
• Adobe application settings are not correct (GPU acceleration not enabled, etc.)

If you continue to have issues, we recommend posting in the comment section below – ideally with a screenshot of the problem and a copy of any log files that are generated by the benchmark.

How does the scoring work?

Changed in PugetBench for After Effects 0.95. The scoring system used in our benchmark is based on the performance relative to a reference system with the following specifications:

• Intel Core i9 11900K 8 Core
• NVIDIA GeForce RTX 3080 10GB
• 64GB of RAM
• Samsung 980 Pro 2TB
• Windows 10 Pro
• Adobe After Effects ver. 22.0

Performance for each test is compared to the reference result, which are then combined with other relevant test results to generated the various scores. The Overall Score is weighted 40% RAM Preview, 40% Render, and 20% Tracking.

Update Log

Version 0.96.0 BETA

• Added preliminary support for After Effects 24 (currently in beta)
• Switched all fonts used in test compositions to defaults found in Windows and MacOS. This should not measurably impact benchmark performance but keeps Creative Cloud from needing to download custom fonts to the user’s system

Version 0.95.7 BETA

• Updated CLI utility to support changes to plugin paths when installing via Creative Cloud

Version 0.95.6 BETA

• Added plugin and CLI support for After Effects 23.0
• Updated benchmark upload/view URLs to match web hosting changes

Version 0.95.2 BETA

• Added filter to remove “(C)”, “(R)”, and “(TM)” strings from the systems specifications
• Added additional granularity to the “Custom” preset. You can now select individual tests to run rather than only being limited to specific groups of tests.
• Added the ability to specify which tests to run with the “Custom” preset when using the CLI
  • Include the “/custom_tests” argument when launching the CLI along with a comma-separated list of test numbers.
  • The test number associated with each test can be seen if you launch the benchmark plugin manually. It will be shown before the name of each selectable test.
  • Example: /custom_tests “8,21”
• Added “/cooldown” CLI argument to specify how many seconds to wait at the end of the benchmark run to allow the system to cool down. By default, the CLI waits for 20 seconds
  • Example: /cooldown 60
• Added the ability to launch custom .bat files at various points during the benchmark run. This was requested multiple times in order to automate additional logging using utilities such as Windows Performance Recorder.
  • If running the benchmark manually, there is now a “Run custom commands” checkbox in the automation settings
  • If using the CLI, include the command line argument of “/custom_tests”
  • Four different .bat files are launched at the start/end of the benchmark, and at the start/end of each individual test.
  • More information on how to make the .bat files, where to save them, and a sample set of files is available in the user guide
  • Note that if the commands require admin privileges, you will need to launch the application or CLI as admin.
• Required application information for the CLI moved to an .ini file that resides alongside the CLI in the plugin folder. This information was previously baked into the CLI itself, but by having it in an editable file, end users can add support for things like beta and pre-release versions of Premiere Pro. The .ini requires the following entries:
  • Main section title: The major version of Premiere Pro. This is used by the CLI for the “/app_version” argument
  • prefLoc: Location of the application preferences folder
  • prefFile: In many cases, this is the same as prefLoc, but some benchmarks need a specific preferences file defined
  • appLoc: Path to the Premiere Pro application
  • appEXE: Name of the Premiere Pro .exe when it has been launched
• Fixed CLI issue where it wasn’t detecting existing preferences files, and always had to launch After Effects twice.

Version 0.95.1 BETA

• Minor bugfix to resolve issues when activating a license key

Version 0.95 BETA (Major Update)

• Due to the changes to the tests and scoring, the results in this version cannot be directly compared to previous benchmark versions
• Added CLI, RAM verification, and settings check support for After Effects version 22.x
• Settings requirement added for disabling “Cache frames when idle” when running Ae version 22.x
  • This is necessary because otherwise After Effects may start rendering frames before the benchmark runs which would throw off the results
• Tracking tests now run 3x each, with the best result being used for scoring
  • Adding loops because tracking can be more inconsistent between runs than preview or rendering
• Overall Score adjusted to be weighted 40% RAM Preview, 40% Render, and 20% Tracking rather than being an even division between the three types of tests
• Added new “Multi-Core Score” to show best-case performance for higher core count CPUs with MFR
  • Similar to how the “GPU Score” shows the largest difference you may see between GPUs in After Effects, the “Multi-Core Score” shows the largest difference you are likely to see with higher core count CPUs
• Significant changes made to the tests – mostly due to the large performance gains experienced with MFR
• Dropped tests:
  • All RAM Preview (Half Res) tests. With MFR, performance may be real-time on some systems, which isn’t desirable for showing the performance delta between hardware
  • C4D Track – Did not provide significantly unique data from the C4D Text composition
• Adjusted tests:
  • Polygon – Increased difficulty by creating 3 sets of polygon pre-comps
  • Glitch – Updated graphic to match new Puget Systems logo and removed tint effect (breaks randomly on some systems for an unknown reason)
  • Realistic Rain – Updated graphic to match new Puget Systems logo
• Added tests:
  • Pulse – Slightly simplified version of the “AE Pulse Benchmark” project provided by Adobe to test multi-frame rendering. Shorted to 100 frames and removed unused layers
  • Gaming Intro – slightly modified version of the “Gaming Intro” motion graphics template that is include in Premiere Pro

Version 0.93.2 BETA

• Added CLI and RAM verification support for After Effects 18.0

Version 0.93 BETA

• Fixed issue with After Effects 14.1.4 that prevented the RAM Preview tests from completing due to only 99 frames out of 100 being rendered.

Version 0.92 BETA

• Moved to using node-wmi to gather system specs on Windows systems. This should be significantly faster and more reliable than the previous command line method
• Added GPU driver and motherboard BIOS to the system specs for Windows systems
• Misc bug fixes

Version 0.91 BETA

• Improved system spec gathering reliability
• System specs on MacOS now gather properly even if the system drive’s name has been changed from the default “Macintosh HD”

Version 0.9 BETA (Major Update)

• Results are now uploaded to our online database. This is required for the free version, but opt-in if you have a commercial license.
• Removed the result screen at the end of the benchmark run now that the full results can be viewed on our benchmark listing page.
• Moved from a script to a full plug-in. This was required for the result uploading, but also allows us much more flexibility for configurable settings and increases consistency between our various Adobe benchmarks.
• License validation moved from the CLI utility to the plugin itself.
• Added the ability to do a “Custom” test which does the same tests as normal, only on whatever compositions are on the root-level of the currently opened project.
• Added ability to set the location for any disk-based tests (rendering).
• Added tooltips for the various settings that can now be configured.
• Status logs and configuration settings moved to “~Documents⧵PugetBench⧵After Effects⧵TIMESTAMP” since we cannot log directly to the plugin folder.
• Removed “Progress” text layer since this information is now reported in the plugin UI.
• Disk Cache is now cleared by directly deleting the cache files. This removed the need to click the “Are you sure you want to clear the cache?” window that used to come up
• General bug fixes and stability improvements.

Version 0.82 BETA

• The Adobe After Effects team helped us figure out most of the improvements in this update. Thank you to the dev team!
• Improved method for starting the RAM Preview tests. This fixes numerous issues with the previous method
• No longer need to manually set spacebar playback settings for “loop once”
• We can now detect when not all frames were played during RAM Preview (due to user intervention or lack of RAM) and can throw an error
• Now able to start the render queue without it looking like AE froze. Due to this, we no longer need to use AERender for the render tests and can use the built-in render queue
• Misc improvements for how we loop through the different tests
• Due to the change to using the render queue, the scores/results in this version may be slightly different than previous versions

Version 0.8 BETA

• Added support for After Effects 2020
• Renamed benchmark to “PugetBench for After Effects”
• Added motherboard and RAM speed (Windows only) to the results page
• Removed .csv log file support in the free edition (log files will be a feature in the commercial use version)
• More code to ensure consistency between benchmark runs
• Miscellaneous bug fixes and improvements

Version 0.52 BETA

• Overall Score rounding error fix. Previous versions were rounding to the nearest 10. I.E. “950” instead of “948”

Version 0.51 BETA

• Added Windows buildnumber to the Benchmark Results page
• Making sure to set the renderer to “Classic 3D” on most comps, or “Cinema 4D” on the C4D comps (just incase it gets changed somehow)
• More robust reporting for render errors such as not enough frames rendering or the render failing completely.
• Clearing out export folder at the start of each run just so that aerender doesn’t have to overwrite files.
• Scoring fix for when a render completes in exactly a minute (the log reports different if it is less than a minute, over a minute, or exactly on a minute)

Version 0.5 BETA (Major Update)

• Switching from “year.mon.day” version format to a more typical versioning scheme so that we can more easily show when things like the scoring has changed enough to not be interchangeable with previous versions.
• Dropping color depth from 32 bpc to 16 bpc. At 32 bpc, you needed more than 20GB of RAM in order to RAM preview the UHD compositions and anything less simply stopped the test early resulting in artificially high scores for that test. Dropping to 16 bpc will allow people with 16GB of RAM to successfully run the benchmark.
• Added a RAM checker that throws an error if you have less than 12GB of RAM available for AE to use. Anything less results in the same issue mentioned in the previous bullet point with artificially high scores.
  • This checker is not very robust at the moment since it has to read some configuration files from /Adobe/dynamiclinkmanager/*version*. I can’t find a way to directly link the version of AE with the version of the DLM, so for now it is hard coded and allows you to keep running the benchmark if it breaks.
• Added a “Progress” text layer on each test comp that updates automatically so you can visually see how far along the benchmark is.
• Export (Final Render) test now uses aerender (background renderer) rather than the render queue built in to After Effects. This is nearly the same performance as using the render queue, but much easier to automate.
  • Added a “status” comp that shows you how far along it is since there is little to no visual feedback that After Effects isn’t just broken
  • This fixes most of the issues we’ve seen with this test failing on some systems
  • This removes the need to do any security setting changes in MacOS (yay!)
  • This also opens up the possibility of adding a render node test in the future that spawns multiple instances of aerender.
• Switching from Overall/Standard/C4D/Tracking scores to Overall/Render/Preview/Tracking to align better with what people are interested in. If someone only cares about C4D performance, they can look at the raw results for those compositions.
• There is now a “Benchmark Results” comp that comes up at the end of the benchmark that displays a bunch of useful information including:
  • Benchmark version
  • Overall & section scores
  • FPS results for each individual test
  • System information like CPU, RAM, OS, GPU, GPU acceleration mode, and After Effects version
• In addition to the log file with the results, the benchmark now also makes a PNG of the “Benchmark Results” comp
• Adjusting benchmark tests:
  • Adding three new compositions:
    • “Expressions” – almost exclusively uses expressions
    • “Motion Blur” – Simply comp with motion blur being the limiting factor
    • “GPU Stress” – our attempt to show the maximum difference you would see with a higher-end GPU
  • Removing old compositions that were either redundant or are tested better with the new comps:
    • “Glitch Project”
    • “Turbulant Noise Keyframe”
    • “Rotation Keyframe”
    • “C4D Text Draft Project”
    • “C4D Track Draft Project”
  • Replaced video media in the “Phone Composite Project”, “Tracking”, and “C4D Track Final Project” with lower bitrate H.264 media. It doesn’t significantly impact the performance, but results in a much smaller benchmark download size.
• Miscellaneous bug fixes and improvements

Version 19.5.16 BETA

• Fixed issue where the benchmark could not verify the loop playback setting on minor revisions of AE.

Version 19.4.4 BETA

• Reordered the tests to Render->Half Res Preview->Full Res Preview in order to try to resolve issues with some systems improperly using the disc cache files for the render test which resulted in too high of scores.
• Added additional checks during the Render tests so that if the benchmark is not able to start the render automatically, it will prompt the user to click the “Render” button manually.

Version 18.12.18 BETA

• Minor bug fix

Version 18.12.17 BETA

• Added more robust detection for when the benchmark starts/stops RAM Preview and Render testing. This is to fix a bug on some systems where if the test doesn’t start within a few seconds, the benchmark thinks it already finished.

Version 18.12.10 BETA

• Added Ae version check to make sure Ae 2019 (ver 16.x) is being used.
• Initial benchmark errors (Disk Cache not disabled, Ae not using English, Ae version mismatch, etc) now allow for you to continue the benchmark anyways – although the benchmark will probably break so that isn’t recommended.

Version 18.12.0 BETA

• First release.