The Doom rendering engine is the core of the game engine that powers Doom and its sequels, and that is used as a base to power other games by id Software licensees, notably Heretic, Hexen, and Strife. It was created by John Carmack, with auxiliary functions written by John Romero, Dave Taylor, and Paul Radek. Originally developed on NeXT computers, it was ported to DOS for Doom's initial release, and later ported to several other operating systems and game consoles.
The source code for the Linux version of the Doom games was released to the public in 1997 under a license that granted rights to non-commercial use, and re-released under the GNU General Public License in 1999. As a result, dozens of user-developed source ports have been created which allow Doom to run on previously unsupported operating systems, often fix bugs (including the static limits noted below), and sometimes radically expand the engine's functionality with new features.
It is not a true "3D" engine (as it is not possible to look up and down properly, and one sector cannot be placed above or beneath another), but is however a fairly elegant system which allows pseudo-3D rendering. When first published, Doom was revolutionary and almost unique in its ability to provide a fast texture-mapped 3D environment on contemporary hardware — late-model 386 and early 486 PCs, without specialised 3D graphics hardware, running at clock speeds of around 25-33MHz.
Despite the simplicity and speed of the renderer, it has limitations. The base renderer relies on 16.16 fixed point numbers (whole numbers between -32,768 and 32,767 with fractions limited to multiples of 1/65,536). Due to such limitations, accuracy in small units is lost as the limited precision hinders accuracy especially when multiplying and dividing. High resolutions cause more graphical glitches especially above the 5,000 pixel resolution range, some glitches appear as field of view distortions along with floors and ceilings extending towards the horizon.
The following is only an overview of the basic structure of a Doom engine level. Most of the data structures listed here carry extra properties, such as texture offsets, or flags for restricting player or monster movement.
Viewed from the top down, all levels are actually two-dimensional, demonstrating one of the key limitations of the engine: it is not possible to have "rooms above rooms". This limitation, however, has a silver lining: a map mode can be easily displayed which represents the walls and the player's position, as shown schematically in the first image to the right (in contrast, the map mode of Doom's close contemporary Descent — which used an unfettered 3D engine  — was extremely hard to interpret).
The base unit is the vertex, which signifies a single 2D point. In the diagram to the right, each small blue square is a vertex. Vertices (or "vertexes" as they are referred to internally) are then joined to form lines, known as linedefs. Each linedef can have either one or two sides, which are known as sidedefs. Sidedefs are then grouped together to form polygons; these are called sectors.
Sectors represent particular 2D areas of the level. Each sector has a number of required properties: a floor height, a ceiling height, a light level, a floor texture, and a ceiling texture. To put two different light levels in the same room, for example, a new sector must be created for the second area. A sector must be completely surrounded by sidedefs; therefore, one-sided linedefs represent solid walls, while two-sided linedefs represent boundary lines between sectors.
Sidedefs are used to store wall textures, which are completely independent of floor and ceiling textures. Each sidedef can have up to three textures; these are called the middle (or "normal"), upper, and lower textures. In one-sided linedefs, only the middle texture is used for the texture on the wall. In two-sided linedefs, the situation is more complex. The upper and lower textures are used to fill the gaps where adjacent sectors have different floor and ceiling heights: lower textures are used for stairs, for instance. Most sidedefs will not have a middle texture, although some do; this is used to make textures "hang" in mid-air. (For example, when a transparent bar texture is seen forming a cage, a middle texture has been used on a two-sided linedef.) An upper or lower texture on a one-sided linedef, or on a two-sided linedef whose adjoining sectors always have the same floor and ceiling heights, is highly unusual and may even cause rendering problems if not handled carefully by the level designer(s).
Finally, there is a list of essentially non-architectural objects in the level, called things. These are used to place players, monsters, powerups, free-standing decorations, obstacles, etc. Each thing is given a 2D coordinate, as with the vertices. Things are then automatically placed on the floor or the ceiling as appropriate to their general type.
The Doom engine makes use of a system known as binary space partitioning (BSP). A tool, known as a node builder, must be used to generate the BSP data for a level before it can be played. Depending on the size and complexity of the level, this process can take quite some time.
BSP divides the level up into a binary tree: each location in the tree is a node which represents a particular area of the level (with the root node representing the entire level). At each branch of the tree there is a dividing line which splits the area of the node into two subnodes. At the same time, the dividing line divides linedefs into line segments called segs.
At the leaves of the tree are convex polygons, where it is not useful to divide the level up any further. These convex polygons are referred to as subsectors (or SSECTORS), and are bound to a particular sector. Each subsector has a list of segs associated with it.
The BSP system is really a very clever way of sorting the subsectors into the right order for rendering. The algorithm is fairly simple:
- Start at the root node.
- Draw the child nodes of this node recursively. The child node closest to the camera is drawn first. (This can be found by looking at which side of a given node's dividing line the camera is on.)
- When a subsector is reached, draw it.
For a given column of pixels, the process is finished when the whole column is filled (i.e., there are no more gaps left). This ordering ensures that during actual gameplay, no time is wasted drawing objects which are not visible; as a result, maps can become very large without any speed penalty.
When a level is revised using an editor, the BSP data must be updated only if a structural change has been made (the level can still be used without rebuilding its nodes, but only non-structural changes will be taken into account by the engine). In this context, structural changes include:
- Adding, deleting, or moving vertices.
- Adding or deleting linedefs.
- Adding or deleting sidedefs.
- Adding or deleting sectors, or changing the sectors with which given sidedefs are associated.
The following are not considered structural changes:
- Any change involving only things.
- Adding or removing wall textures, or replacing one texture with another.
- Changing the actions of linedefs or sectors, or rearranging tags.
- Changing floor or ceiling heights.
The Doom engine renders the walls as it traverses the BSP tree, drawing subsectors by order of distance from the camera (that is, the closest segs are drawn first). As the segs are drawn, they are stored in a linked list. This is used to clip other segs rendered later on, reducing overdraw. The list is also used later to clip the edges of sprites. To reduce the burden on the engine's performance, which was essential back in the early 90s when the game was developed and released, there is a static limit to how many segs may be rendered at once, which is 256. The excess segs are simply not drawn, leaving visible gaps where walls should be, and creating a hall of mirrors effect. Fortunately, because of the order in which segs are drawn, the gaps are out in the distance where they are less noticeable.
Once the engine reaches a solid (one-sided) wall at a particular X coordinate, no more lines need to be drawn at that area. As this occurs, the engine builds up a "map" of areas of the screen where solid walls have been reached. This allows distant parts of the level, currently invisible to the player, to be clipped completely.
Wall textures in WADs are stored as sets of vertical columns; this is useful to the renderer, which essentially renders the walls by drawing many vertical columns of texturing. Because all walls are drawn vertically, the player cannot properly look up or down. It is possible to approximate looking up and down via "Y-shearing", as the Heretic engine and some source ports do. Essentially this works by increasing the vertical resolution, and then defining a "window" on that space. As the window moves up or down, it creates the illusion that the player is looking up or down. However, the view tends to become increasingly distorted as the player looks further and further away from the horizon.
Drawing floors and ceilings
The system for drawing floors and ceilings ("flats") is less elegant than that used for the walls. Flats are drawn with a flood fill-like algorithm. Because of this, it is sometimes possible (if a buggy node builder has been used) to see "holes" where the floor or ceiling texture bleeds out to the edge of the screen. This is also the reason that, if the player travels outside the level proper using the no-clipping cheat code, the floors and ceilings will appear to stretch out from the nearest room(s) through the empty space.
Floors and ceilings are drawn as "visplanes", which represent horizontal runs of texture, given a floor or ceiling at a particular height, light level and texture (if two adjacent sectors have the exact same floor, these can be merged into one visplane). Each X position in the visplane has a particular vertical line of texture which is to be drawn.
Because of the requirement to draw only one vertical line at each X position, it is sometimes necessary to split one visplane into multiple visplanes. For example, consider looking at a floor with two concentric squares. The inner square will vertically divide the surrounding floor, and within the horizontal range of the inner square, two visplanes are needed for the surrounding floor.
To curtail system-taxing level designs, the engine's programmers added a static limit of 128 simultaneous visplanes. This later frustrated any fan community level designers attempting large or detailed levels for a time until the source was released, which allowed modifications raising or removing the limit. If the limit is exceeded, the game immediately exits to DOS with the message, "No more visplanes". One easy way to invoke the visplane limit is a large checkerboard floor pattern, which creates a considerable number of visplanes when viewed.kjl
As the segs are rendered, visplanes are also added, running from the edges of the segs towards the vertical edges of the screen. These are extended until they reach existing visplanes. This system is dependent on the segs being rendered in order by the engine; it is necessary to draw nearby visplanes first, so that they can be "cut off" by the more distant ones. If unclipped, the floor or ceiling will "bleed" toward the edge of the screen, as previously described. Eventually, the visplanes form a "map" of particular areas of the screen in which to draw particular textures.
While visplanes are constructed essentially from vertical "strips", the actual low-level rendering is performed in the form of horizontal "spans" of texture. After all visplanes have been constructed, they are converted into spans which are then rendered to the screen. This appears to be a tradeoff: computation time is saved by constructing visplanes as vertical strips, but in order to decrease the number of perspective calculations made, it is easier to draw them as horizontal strips. Just as a single vertical strip of wall is at a constant distance from the camera, a single horizontal strip of a floor or ceiling is at a constant distance, and so some rendering calculations can be done once and used for an entire span.
Drawing things (sprites)
Each sector within the level has a linked list of things stored in that sector. As each sector is drawn, the sprites are placed into a list of sprites to be drawn. Sprites outside the field of view are ignored.
The edges of sprites are clipped by checking the list of segs previously drawn. Doom engine sprites are stored in the same column-based format as wall textures, which again is useful for the renderer: the same functions may be used to draw both walls and sprites.
While subsectors are guaranteed to be in order, the sprites within them are not. The engine stores a list of sprites to be drawn ("vissprites"), and sorts this list before rendering any things. Distant sprites are drawn before nearby ones; this causes some overdraw, but it is usually negligible. The number of sprites allowed to be drawn at once is limited to a maximum of 128. If there are more than 128 sprites in view at a time, the excess amounts are simply not drawn, which gives the effect of things popping in and out of existence, with greater numbers of sprites disappearing if correspondingly larger numbers of things are present.
There is a final issue involving middle textures on two-sided linedefs, parts of which may be transparent (and therefore fail to clip sprites as another wall would). Such textures are listed and drawn with the sprites at the end of the rendering process, rather than with the rest of the walls. This alternate drawing scheme is why the Medusa effect applies only to middle textures on two-sided linedefs.
- The Doom source code, released in 1999 under the GNU General Public License
- BSP Frequently Asked Questions
- GL nodes specification
- Simple graphical view of the BSP algorithm
- Another programmer, Michael Abrash, appears in the credits screen for The Ultimate Doom, although he states he joined id Software to work on Quake in early march of 1995.  However, David Kushner's Masters of Doom does note (p. 189) that the id programmers read Abrash's published works on graphics programming with attention while they developed their earlier games.