Painless Optimization Guide

This guide aims at explaining how triangles and portals are generated, so you can understand how come you get some much stuff rendered in that one place in your map, and how not to render that much. There might not be "one magic trick" to solve each and every rendering issue you may face, but if you understand how it's happening, you also know how to prevent it.

Although this document was written with Kingpin in mind, it should apply to all games based on the Quake II engine (Quake II, Kingpin, Soldier of Fortune, SiN, Daikatana, and probably Anachronox too). Note that some things may differ slightly from one game to the other (maybe not all is applicable to Soldier of Fortune for instance), and from utilities to utilities (some forks include bug fixes, custom features, and everything in-between). For this guide, QBSP3 (Revision 1.09 by Geoffrey DeWan), QVIS (Revision 1.03 by Geoffrey DeWan), and ArghRad (Version 2.01 by Tim Wright) were used.

Compilation process

KPBSP and KPVIS

KPBSP's primary purpose is to output a "barebones" .BSP file out of a source .MAP file; Amongst many things, KPBSP...

Portals, portal clusters, visibility

Most map designers know the graphic engine doesn't draw the whole level each frame: it selects only a few chunks of the map to render, based on the location of the camera and the Visibility Information Set (it also culls surfaces depending on their orientation and camera's direction, but it's unrelated to the topic at hand). It is also widely accepted that the fewer triangles are being drawn, the faster the rendering process becomes. Now, in order to help the VIS creation, we need to understand how it is generated. We know that KPBSP generates portals, and we know that all KPBSP requires is a simple map file; it means that portals have to be generated from brushes, right?

Brushes and planes

You certainly already know that brushes have to be convex, but maybe you don't know why. The reason is: brushes are internally described as the volume defined by the intersection of multiple planes. It sounds complex, but it really isn't. A "plane" is to a "face" what a "line" is to a "line segment": planes and lines are infinite, faces and line segments are finite.

This can be easily tested by creating a simple box in Radiant, and "removing" (or "skipping") one of the plane by setting its surface flag to "skip" (flag: 512). After compilation, the box will extend in the direction of the missing plane (remove one plane and the volume will "leak through"). In other words, planes work exactly like Radiant's 3-point clipping tool.

Planes and portals

Portals are the "cut off" part of the plane, or the extension of volumes' surfaces. They are like infinitely thin windows splitting up the empty inner space of the level into smaller convex volumes called portal clusters (sometimes also referred to as "Leaf", "VisLeaf", or "Cluster"). Note that portals can split each other, but won't cross each-other: they will only extend until they hit a solid surface or another portal. They also only exist within the inner space of the level; they will never overlap solid surfaces.

NB: Portals are generated following a strict set of rules: KPVIS will prioritize axial portals (parallel to the X, Y or Z axis), then focus on portals that will create fewer splits in clusters.

Portals and portal clusters

Portal clusters are empty volumes defined by portals and/or solid surfaces. They are like the "non-solid" counterpart of brushes. Interestingly, like brushes, portal clusters are always convex, and their portals always face "outward", which means that portals belonging to the same cluster are "back-to-back" and cannot see each other... that is, until they do due to multiple splits (and the loss of precision in their new coordinates), in which case KPVIS throws the infamous "portal saw into leaf" warning.

Peek-a-boo

Basically, the rendering engine will rely on the VIS to know what cluster must be drawn according to the cluster the player is standing in. The table is built by determining the visibility between portals rather than clusters. Simply, if two portals can see each other, then the cluster they belong to can also see each other. Visibility is determined by tracing a ray from any two portals. If the ray doesn't collide with any solid surface, then these two portals can see each other.

It doesn't matter if the two portals are not completely visible to one another, or if only a part of the cluster is visible. All it takes is one successful peek from one portal to another. The easiest way to guarantee that two portals cannot see each other is to make sure they are joining in a straight (=180°) or reflex angle (>180°). There's also an easy trick to properly place hint brushes, which will be explained in a few parapraphs.

Hiding more triangles

Designing line-of-sight breakers

Obviously, you cannot hide part of the geometry if you don't put a wall to occlude it (that's why open areas are such a pain to optimize), and even seemingly "closed" environments may need extra occlusion. That's why Xatrix often designed convoluted tunnels, hallways, and ventilation ducts that could have been much easier to navigate.

These passages are often U-shaped to break line of sight and conceal areas from each other, thus rendering smaller chunks of the level at once. A shorter variation of U-shaped hallways is a simple corridor with two doors located on the same wall: since both doors are facing the same direction, they cannot see each other, thus breaking line of sight. From a gameplay perspective, breaking line of sight is also a good thing as it allows the player to take cover and escape attackers.

If you have designed a room with tons of details, or if you're coming from a large outdoor area, let the door open directly onto a wall. This also works for windows by the way.

Using func_areaportal

Kingpin will always attempt to draw rooms located behind closed doors because brush entities (like func_door) do not generate portals and thus cannot block line of sight. However, when it comes to func_door or func_explosive, it is still possible to force the engine to not render the area behind by using func_areaportal.

Insert a func_areaportal entity inside the func_door entity, making sure that it covers the whole height and length of the hole. Then, link func_door to func_areaportal via their respective "target" and "targetname" fields. Note that func_areaportal can be triggered with func_button or other mechanisms if you can make use of it.

Placing hint brushes

Hint brushes are made of multiple skip surfaces (flag: 512) and one (or more) hint surface (flag: 256), which will be interpreted by VIS as a portal. In other words, the brush itself doesn't matter, what's important is its hint surface(s).

Here's an easy trick to always place your portals correctly: think of hint surfaces as a line of sight. Draw a line that cuts the whole area in two, and make sure it includes the camera position and the edge of the occluding wall in its path. That line is where you want to put your hint surface. Never trust KPBSP to create portals where you expect them to be and always extend the surface as far as necessary.

Remember that if you can trace a ray from one hint surface to another, they can see each other; also, co-planar hint surfaces cannot see each other.

Reducing triangle count

Reducing triangle mesh subdivision by avoiding surface contact

When two brushes belonging to the same entity touch, they are merged into one triangle mesh by KPBSP, usually creating extra triangles in the process. However, different entities never share their triangles' edges, and worldspawn is no exception. KPBSP will create submodels specifically for each brush entity (func_door, func_wall, trigger_hurt, etc.) This means that brush entities will not generate triangle subdivisions when touching each other.

Xatrix often relied on mesh isolation by turning complex props into func_wall brush entities. This can be observed on barrels, pipes, signs, table legs, etc. In other cases, props were not turned into brush entities: in "bar_sr" for instance, instead of making the two sinks of the ladies' bathroom func_wall, it's the wall against which they are placed that was turned into a brush entity and embedded inside a narrow alcove. Using brush entities has its own downsides however:

Brush entities use a slightly different lighting method than worldspawn, which may contrast with its surroundings.
Brush entities won't cast shadows. While there's no workaround for mobile entities (like func_door), you may consider adding detail structural brushes inside func_wall entities to mimic the standard behavior, or (in last resort) rely on negative lights to manually draw shadows.
Entities are rendered in the order in which they are created. This is an issue for brush entities using transparent surfaces, as some entities may wrongly appear "in front" of others.
Brush entities cannot block line of sight and are ignored by KPVIS.

To avoid surface contact, some level designers opted for the most straightforward approach possible: offsetting props by one unit. This technique usually prevents the creation of extra triangles, but may sometimes produce weird shadows nearby. The one-unit offset is almost impossible to notice when used on hanging lights and see-through elements such as barbwire, link fences, and neon signs.

Upscaling textures

The lightmap resolution and the triangle count required to draw a surface depend on its texture scale: a surface will require fewer triangles if its texture is upscaled (above 1.0) and will need more triangles if its texture is downscaled (below 1.0).

Consider abusing texture scaling for large outdoor areas, but keep in mind that the lightmap's resolution will be lowered, which will negatively impact baked lights (KPRAD) and result in odd dynamic lights (although upscaling textures on surfaces that shouldn't normally be visible is a good thing as it reduces light patches usage). The surface will also look blurrier so you might want to reserve this trick for distant objects. Find textures you can stretch in such a way it won't be too noticeable, increase horizontal scale of horizontal wood beams for instance.

NB: Rotated textures may result in a more complex triangle mesh, depending on the surrounding architecture.

Embracing the 1K units cuts

Starting at origin (0, 0), KPBSP will slice the static geometry (worldspawn) every 1024 units on the X and Y axis, thus creating independent blocks; surfaces crossing the subsequent 1024 unit marks (1024, 2048, 3072, 4096, etc.) are split into additional triangles. Extra portals are also automatically generated, and portals created by structural brushes of neighboring blocks are ignored.

Simply moving the whole map around the grid can increase (or decrease) the poly count and affect KPVIS. Redesigning parts of the map to align doorframes, corners and walls on these cuts in order to avoid useless geometry subdivision is worth the time and effort. When done properly, this will not only reduce the number of triangles in a scene, but also reduce the number of portals and portal clusters in a map, thus boosting KPVIS processing time.

Removing surfaces

You may remove a surface by applying surface properties Sky (flag: 4) and NoDraw (flag: 128) to it. On the plus side, this will free some light patches. On the other hand, the surface will be "missing" and will display the skybox instead (which doesn't technically appear on the poly count, but still needs to be drawn). Eitherway, surfaces are automatically culled on the fly according to the camera position (regardless of the Visibility Information Set), so it may only be useful in some very specific situations such as backdrop elements located in map surroundings.

Lowering the level of detail

Before removing props altogether, you may try simplify their shape; use fewer faces for arches, pipes, barrels, etc. Aim for 600 wpoly during building stage (you're likely to exceed this count, but limiting yourself sooner will make things easier in the future), and enforce a strict below 800 wpoly count when the map is complete (at that point, you might still pull it off with a few hint brushes).

Extras

Useful scripts

To toggle display triangles, the following command lines can be stuffed into autoexec.cfg:

alias glst_on "gl_ext_multitexture 0; gl_showtris 1; vid_restart; r_speeds 1; alias glshowtris glst_off;"
alias glst_off "gl_ext_multitexture 1; gl_showtris 0; vid_restart; r_speeds 0; alias glshowtris glst_on;"
alias glshowtris "glst_on"
bind v glshowtris

Press "v" anytime to show/hide triangles.

Not to be confused with "r_novis", "gl_lockpvs" stops updating visibility: regardless of where the player is located, areas that were visible when enabling gl_lockpvs remain visible, while concealed areas remain invisible.