The British National Corpus (BNC) is a 100-million-word text corpus of samples of written and spoken English from a wide range of sources. The corpus covers British English of the late 20th century from a wide variety of genres, with the intention that it be a representative sample of spoken and written British English of that time. It is used in corpus linguistics for analysis of corpora.
History The project to create the BNC involved the collaboration of three publishers (with the Oxford University Press as the lead collaborator, Longman and W. & R. Chambers), two universities (the University of Oxford and Lancaster University), and the British Library. The creation of the BNC started in 1991 under the management of the BNC consortium, and the project was finished by 1994. There have been no additions of new samples after 1994, but the BNC underwent slight revisions before the release of the second edition BNC World (2001) and the third edition BNC XML Edition (2007).What is the BNC?. Retrieved 12 March 2012.
The BNC was the vision of computational linguists, whose goal was a corpus of modern (at the time of building the corpus), naturally occurring language in the form of speech and text or writing that could be analysed by a computer. Hence, it was compiled as a general corpus to pave the way for automatic search and processing in the field of corpus linguistics. One of the ways the BNC was to be differentiated from existing corpora at that time was to open up the data not just to academic research, but also to commercial and educational uses.
The corpus was restricted to just British English, and was not extended to cover World Englishes. This was partly because a significant portion of the cost of the project was being funded by the British government which was logically interested in supporting documentation of its own linguistic variety. Because of its potentially unprecedented size, the BNC required funds from the commercial and academic institutions as well. In turn, BNC data then became available for commercial and academic research.
Description The BNC is a monolingual corpus, as it records samples of language use in British English only, although occasionally words and phrases from other languages may also be present. It is a synchronic corpus, as only language use from the late 20th century is represented; the BNC is not meant to be a historical record of the development of British English over the ages. From the beginning, those involved in the gathering of written data sought to make the BNC a balanced corpus, and hence looked for data in various mediums.
Components and content
90% of the BNC is samples of written corpus use. These samples were extracted from regional and national newspapers, published research journals or periodicals from various academic fields, fiction and non-fiction books, other published material, and unpublished material such as leaflets, brochures, letters, essays written by students of differing academic levels, speeches, scripts, and many other types of texts.British National Corpus. Retrieved 12 March 2012.
The remaining 10% of the BNC is samples of spoken language use. These are presented and recorded in the form of orthographic transcriptions. The spoken corpus consists of two parts: one part is demographic, containing the transcriptions of spontaneous natural conversations produced by volunteers of various age groups, social classes and originating from different regions. These conversations were produced in different situations, including formal business or government meetings to conversations on radio shows and phone-ins. These were to account for both the demographic distribution of spoken language and those of linguistically significant variation due to context.
The other part involves context-governed samples such as transcriptions of recordings made at specific types of meeting and event. All the original recordings transcribed for inclusion in the BNC have been deposited at the British Library Sound Archive. The majority of the recordings are freely available from the Oxford University Phonetics Laboratory.
Sub-corpora and tagging Two sub-corpora (subsets of the BNC data) have been released: BNC Baby and BNC Sampler. Both these sub-corpora may be ordered online via the BNC webpage. BNC Baby is a sub-corpus of BNC that consists of four sets of samples, each containing one million words tagged as they are in BNC itself. The words in each sample set correspond to a specific genre label. One sample set contains spoken conversation and the other three sample sets contain written text: academic writing, fiction and newspapers respectively. The latest (third) edition has been released and comes in XML format. The BNC Sampler is a two-part sub-corpora, a part each for written and spoken data; each part contains one million words. The BNC Sampler was originally used in a project to work out how to improve the tagging process for the BNC, which eventually led to the BNC World edition. Throughout the project, the BNC Sampler was improved with increasing expertise and knowledge for tagging to arrive at its current form.
The BNC corpus has been tagged for grammatical information (part of speech). The tagging system, named CLAWS, went through improvements to yield the latest CLAWS4 system, which is used for tagging the BNC. CLAWS1 was based on a hidden Markov model and, when employed in automatic tagging, managed to successfully tag 96% to 97% of each text analysed. CLAWS1 was upgraded to CLAWS2 by removing the need for manual processing to prepare the texts for automatic tagging. The latest version, CLAWS4, includes improvements such as more powerful word-sense disambiguation (WSD) abilities, and the ability to deal with variation in orthography and markup language. Later work on the tagging system looked at increasing the success rates in automatic tagging and reducing the work needed for manual processing, while maintaining effectiveness and efficiency by introducing software to replace some of the manual work. Subsequently, a new program called the "Template Tagger" was introduced for a corrective function. Tags indicating ambiguity were later added. Manual tagging is still necessary, as CLAWS4 is still unable to deal with foreign words.
TEI and access The corpus is marked up following the recommendations of the Text Encoding Initiative (TEI) and includes full linguistic annotation and contextual information. The licence for the CLAWS4 part-of-speech tagger may be purchased to use the tagger. Alternatively, a tagging service is offered at Lancaster University. The BNC itself may be ordered with either a personal or institutional license. The edition available is the BNC XML edition and it comes with the Xaira search engine software. Ordering may be carried out via the BNC website. An online corpus manager, BNCweb, has been developed for the BNC XML edition. The interface is designed to be easy to use, and the program offers query features and functions for corpus analysis. Users can retrieve results and data from searches and analyses.
Permission issues The BNC was the first text corpus of its size to be made widely available. This could be attributed to the standard forms of agreement, between rights owners and the Consortium on the one hand, and between corpus users and the Consortium on the other. Intellectual property rights owners were sought for their agreement with the standard licence, including willingness to incorporate their materials in the corpus without any fees. This arrangement may have been facilitated by the originality of the concept and the prominence associated with the project. However, it was a challenge to keep the identity of contributors hidden without discrediting the value of their work. Any distinct allusion to the identity of contributors was largely removed; the alternative solution of substituting the identity of a contributor with a different name was discussed, but not considered feasible.
Additionally, contributors had earlier been asked only to incorporate transcribed versions of their speech and not the speech itself. While permission could be sought from initial contributors again, the lack of success in the anonymization process meant that it would be challenging to seek materials from initial contributors. At the same time, two factors compounded the unwillingness of rights owners to donate their materials: full texts were to be excluded, and there was no motivation for them to disseminate information using the corpus, particularly since the corpus operates on a non-commercial basis.
Problems and limitations
Categories By 2001, the BNC still had no text categorisation for written texts beyond that of domain, and no categorisation for spoken texts except by context and demographic or socio-economic classes. For example, a wide variety of imaginative texts (novels, short stories, poems, and drama scripts) were included in the BNC, but such inclusions were deemed useless as researchers were unable to easily retrieve the subgenres on which they wanted to work (e.g., poetry). Because this metadata was omitted in the file headers and in all BNC documentation, there was no way to know whether an "imaginative" text actually came from a novel, a short story, a drama script or a collection of poems unless the title actually included words such as "novel" or "poem".
With the 2002 introduction of a new version, the BNC World Edition, BNC attempted to deal with this problem. Besides domain, there are now 70 categories for genre for both spoken and written data, and so researchers can now specifically retrieve texts by genre. Even after these additions, however, implementation is still tricky, as assigning a genre or subgenre to a text is not straightforward. The divisions are less clear for spoken data than they are for written data, as there was more variation in topic and execution. Also, there will always be possible subsets of genres of each subgenre. How far genres are subdivided is pre-determined for the sake of a default, but researchers have the option of making the divisions more general or specific according to their needs. Categorisation is also a problem, as certain texts, while deemed to belong to an interdisciplinary genre such as linguistics, include content that is subsequently categorised into either arts or science categories due to the nature of their content.
Classification and discourse Some texts were classified under the wrong category, usually because of a misleading title. Users cannot always rely on the titles of the files as indications of their real content: For example, many texts with "lecture" in their title are actually classroom discussions or tutorial seminars involving a very small group of people, or were popular lectures (addressed to a general audience rather than to students at an institution of higher learning). One reason is that genre and subgenre labels can only be assigned for the majority of the texts in a category. There are subgenres within genres, and for each text the content may not be uniform throughout and may span multiple subgenres. Also, production pressures coupled with insufficient information led to hasty decisions, resulting in inaccuracy and inconsistency in records.
The proportion of written to spoken material in the BNC is 10:1, making spoken material under-represented. This is because the cost of collecting and transcribing one million words of naturally occurring speech is at least 10 times higher than the cost of adding another million words of newspaper text. Some linguists have argued that this represents a deficiency in the corpus, since speech and writing are both equally important in a language. The BNC is not ideal for the study of many features of spoken discourse, since most of its transcripts are orthographic. Paralinguistic features are only roughly indicated.
Limitations and misappropriates