In theoretical physics, the BRST formalism, or BRST quantization (where the BRST refers to the last names of Carlo Becchi, Alain Rouet, Raymond Stora and Igor Tyutin) denotes a relatively rigorous mathematical approach to quantizing a field theory with a gauge symmetry. Quantization rules in earlier quantum field theory (QFT) frameworks resembled "prescriptions" or "heuristics" more than proofs, especially in non-abelian QFT, where the use of "ghost fields" with superficially bizarre properties is almost unavoidable for technical reasons related to renormalization and anomaly cancellation.

The BRST global supersymmetry introduced in the mid-1970s was quickly understood to rationalize the introduction of these Faddeev–Popov ghosts and their exclusion from "physical" asymptotic states when performing QFT calculations. Crucially, this symmetry of the path integral is preserved in loop order, and thus prevents introduction of counterterms which might spoil renormalizability of gauge theories. Work by other authors a few years later related the BRST operator to the existence of a rigorous alternative to path integrals when quantizing a gauge theory.

Only in the late 1970s, when QFT was reformulated in fiber bundle language for application to problems in the topology of low-dimensional manifolds (topological quantum field theory), did it become apparent that the BRST "transformation" is fundamentally geometrical in character. In this light, "BRST quantization" becomes more than an alternate way to arrive at anomaly-cancelling ghosts. It is a different perspective on what the ghost fields represent, why the Faddeev–Popov method works, and how it is related to the use of Hamiltonian mechanics to construct a perturbative framework. The relationship between gauge invariance and "BRST invariance" forces the choice of a Hamiltonian system whose states are composed of "particles" according to the rules familiar from the canonical quantization formalism. This esoteric consistency condition therefore comes quite close to explaining how quanta and fermions arise in physics to begin with.

In certain cases, notably gravity and supergravity, BRST must be superseded by a more general formalism, the Batalin–Vilkovisky formalism.

Technical summary BRST quantization is a differential geometric approach to performing consistent, anomaly-free perturbative calculations in a non-abelian gauge theory. The analytical form of the BRST "transformation" and its relevance to renormalization and anomaly cancellation were described by Carlo Maria Becchi, Alain Rouet, and Raymond Stora in a series of papers culminating in the 1976 "Renormalization of gauge theories". The equivalent transformation and many of its properties were independently discovered by Igor Viktorovich Tyutin. Its significance for rigorous canonical quantization of a Yang–Mills theory and its correct application to the Fock space of instantaneous field configurations were elucidated by Taichiro Kugo and Izumi Ojima. Later work by many authors, notably Thomas Schücker and Edward Witten, has clarified the geometric significance of the BRST operator and related fields and emphasized its importance to topological quantum field theory and string theory.

In the BRST approach, one selects a perturbation-friendly gauge fixing procedure for the action principle of a gauge theory using the differential geometry of the gauge bundle on which the field theory lives. One then quantizes the theory to obtain a Hamiltonian system in the interaction picture in such a way that the "unphysical" fields introduced by the gauge fixing procedure resolve gauge anomalies without appearing in the asymptotic states of the theory. The result is a set of Feynman rules for use in a Dyson series perturbative expansion of the S-matrix which guarantee that it is unitary and renormalizable at each loop order—in short, a coherent approximation technique for making physical predictions about the results of scattering experiments.

Classical BRST

This is related to a supersymplectic manifold where pure operators are graded by integral ghost numbers and we have a BRST cohomology.

Gauge transformations From a practical perspective, a quantum field theory consists of an action principle and a set of procedures for performing perturbative calculations. There are other kinds of "sanity checks" that can be performed on a quantum field theory to determine whether it fits qualitative phenomena such as quark confinement and asymptotic freedom. However, most of the predictive successes of quantum field theory, from quantum electrodynamics to the present day, have been quantified by matching S-matrix calculations against the results of scattering experiments.

In the early days of QFT, one would have had to say that the quantization and renormalization prescriptions were as much part of the model as the Lagrangian density, especially when they relied on the powerful but mathematically ill-defined path integral formalism. It quickly became clear that QED was almost "magical" in its relative tractability, and that most of the ways that one might imagine extending it would not produce rational calculations. However, one class of field theories remained promising: gauge theories, in which the objects in the theory represent equivalence classes of physically indistinguishable field configurations, any two of which are related by a gauge transformation. This generalizes the QED idea of a local change of phase to a more complicated Lie group.

QED itself is a gauge theory, as is general relativity, although the latter has proven resistant to quantization so far, for reasons related to renormalization. Another class of gauge theories with a non-Abelian gauge group, beginning with Yang–Mills theory, became amenable to quantization in the late 1960s and early 1970s, largely due to the work of Ludwig D. Faddeev, Victor Popov, Bryce DeWitt, and Gerardus 't Hooft. However, they remained very difficult to work with until the introduction of the BRST method. The BRST method provided the calculation techniques and renormalizability proofs needed to extract accurate results from both "unbroken" Yang–Mills theories and those in which the Higgs mechanism leads to spontaneous symmetry breaking. Representatives of these two types of Yang–Mills systems—quantum chromodynamics and electroweak theory—appear in the Standard Model of particle physics.

It has proven rather more difficult to prove the existence of non-Abelian quantum field theory in a rigorous sense than to obtain accurate predictions using semi-heuristic calculation schemes. This is because analyzing a quantum field theory requires two mathematically interlocked perspectives: a Lagrangian system based on the action functional, composed of fields with distinct values at each point in spacetime and local operators which act on them, and a Hamiltonian system in the Dirac picture, composed of states which characterize the entire system at a given time and field operators which act on them. What makes this so difficult in a gauge theory is that the objects of the theory are not really local fields on spacetime; they are right-invariant local fields on the principal gauge bundle, and different local sections through a portion of the gauge bundle, related by passive transformations, produce different Dirac pictures.

What is more, a description of the system as a whole in terms of a set of fields contains many redundant degrees of freedom; the distinct configurations of the theory are equivalence classes of field configurations, so that two descriptions which are related to one another by a gauge transformation are also really the same physical configuration. The "solutions" of a quantized gauge theory exist not in a straightforward space of fields with values at every point in spacetime but in a quotient space (or cohomology) whose elements are equivalence classes of field configurations. Hiding in the BRST formalism is a system for parameterizing the variations associated with all possible active gauge transformations and correctly accounting for their physical irrelevance during the conversion of a Lagrangian system to a Hamiltonian system.

Gauge fixing and perturbation theory The principle of gauge invariance is essential to constructing a workable quantum field theory. But it is generally not feasible to perform a perturbative calculation in a gauge theory without first "fixing the gauge"—adding terms to the Lagrangian density of the action principle which "break the gauge symmetry" to suppress these "unphysical" degrees of freedom. The idea of gauge fixing goes back to the Lorenz gauge approach to electromagnetism, which suppresses most of the excess degrees of freedom in the four-potential while retaining manifest Lorentz invariance. The Lorenz gauge is a great simplification relative to Maxwell's field-strength approach to classical electrodynamics, and illustrates why it is useful to deal with excess degrees of freedom in the representation of the objects in a theory at the Lagrangian stage, before passing over to Hamiltonian mechanics via the Legendre transformation.

The Hamiltonian density is related to the Lie derivative of the Lagrangian density with respect to a unit timelike horizontal vector field on the gauge bundle. In a quantum mechanical context it is conventionally rescaled by a factor . Integrating it by parts over a spacelike cross section recovers the form of the integrand familiar from canonical quantization. Because the definition of the Hamiltonian involves a unit time vector field on the base space, a horizontal lift to the bundle space, and a spacelike surface "normal" (in the Minkowski metric) to the unit time vector field at each point on the base manifold, it is dependent both on the connection and the choice of Lorentz frame, and is far from being globally defined. But it is an essential ingredient in the perturbative framework of quantum field theory, into which the quantized Hamiltonian enters via the Dyson series.

For perturbative purposes, we gather the configuration of all the fields of our theory on an entire three-dimensional horizontal spacelike cross section of P into one object (a Fock state), and then describe the "evolution" of this state over time using the interaction picture. The Fock space is spanned by the multi-particle eigenstates of the "unperturbed" or "non-interaction" portion of the Hamiltonian . Hence the instantaneous description of any Fock state is a complex-amplitude-weighted sum of eigenstates of . In the interaction picture, we relate Fock states at different times by prescribing that each eigenstate of the unperturbed Hamiltonian experiences a constant rate of phase rotation proportional to its energy (the corresponding eigenvalue of the unperturbed Hamiltonian).

Hence, in the zero-order approximation, the set of weights characterizing a Fock state does not change over time, but the corresponding field configuration does. In higher approximations, the weights also change; collider experiments in high-energy physics amount to measurements of the rate of change in these weights (or rather integrals of them over distributions representing uncertainty in the initial and final conditions of a scattering event). The Dyson series captures the effect of the discrepancy between and the true Hamiltonian , in the form of a power series in the coupling constant g; it is the principal tool for making quantitative predictions from a quantum field theory.