Apart from natural and many social sciences, population science draws its evidence primarily from statistical data. The information sources of demographic research have traditionally included population census and vital registration, the postwar period has added to the latter sample surveys. In other words, analytical capacity of population science is to a large extent determined by the availability but no less importantly the quality of information collected in the framework of national statistical system.

In case of Estonia, the geopolitical changes that followed the Second World War and the incorporation into the Soviet Union brought the development of national statistical system, which over a period of two decades made significant progress and reached high quality standards, to an end. National statistical institutions were dismantled and replaced by a subordinate branch charged with the implementation of administrative instructions from beyond the country. Statistical methodology introduced under new regime appeared inconsistent with international recommendations, thereby disrupting the constructed series of population data. Moreover, the access to population was severely restricted, making demographic research virtually impossible.

The restoration of national independence in 1991 implied the task of bridging the half-century gap in national statistical system, and following the model of NPI, the Institute became closely involved. Although activities in the referred direction have required a lot of time and effort, often extending from research and methodological work to practical implementation of developed methodology, close involvement has proven unavoidable to secure an appropriate database for demographic research. At the same time, the Institute has contributed to the development of statistical system which forms an essential societal outcome of research activities.

Research under the theme has been divided between two subthemes and a thematic activity related to coordination, application of results and dissemination:

• Population data harmonisation
• Integrated survey statistics
• Estonian population databank

3.1. Population data harmonisation

3.1.1. Background

Following the ultimate cessation of data restrictions in 1991, it became apparent that available population information is heavily deficient, and under existing conditions, of almost worthless for research. In this respect, the reality turned to be quite different from the view, cultivated by the secrecy of population statistics, that the problems of data were mostly, or even exclusively, restrained to availability. From scientific point of view, the major complication of existing population data turned to be the lack of comparability. Being a complex issue, in case of Estonia the lack of comparability involves at least three relatively independent aspects.

First, population data was poorly comparable with other countries. The concepts, definitions, classifications, and in particular the procedures used in data collection differed from those recommended by international organisations and used elsewhere in the world. Often the methodological discrepancies have been not explicit, and under deficient documentation they could easily go unnoticed.

The second aspect of poor data quality is related to temporal comparability. Following the time horizon set by central planning, compilation of population information was oriented strongly biased towards short-term perspective. Data on earlier (longer) periods were usually considered as old and of little interest, and respectively, no efforts were invested into the development of consistent time series, including the very core population indicators, such as population number and age structure.

The third major deficiency relates to subnational/regional data. Due to overcentralisation of decision-making, little if any, information was produced on smaller administrative units, reduction of information started from county level and on communities virtually no information was available, due to applied procedures the figures on regional units not rarely failed to make up national total. As the concept of statistical territorial units was never applied, under frequent administrative transfers it was impossible to develop consistent regional time series etc.

Series of discussions held in the beginning of the 1990s resulted in a conclusion that the information in existing quality cannot serve neither for demographic research nor for the needs of an independent nation in general. To meet the task, a special programme was initiated, which unites multivarious tasks to harmonise the national population statistics since the WWII and bridge the gap with prewar period.

The harmonisation programme of Estonian population data aims at reintroduction of international definitions and classifications, building consistent time series, securing comparability at regional level and integrating vital, census and survey statistics. Accomplishing these tasks includes reviewal and harmonisation of aggregated data, if available, recoding and re-processing of microdata, if endured, and computerisation of archived vital and census records, if aggregated and/or microdata is not available.

The tasks under the programme have carried out exclusively by the Institute, Statistical Office has restricted its responsibility to current production of population data. The harmonisation programme is accomplished through specifically focused sub-projects, the results of the projects are published in publication series of the Institute. Harmonised population data are assembled into Estonian Population Databank (ERA), maintained and operated at the Institute, and disseminated to national as well as international users.

3.1.2. Subtheme frameworks

3.1.2.1. The principal project frameworks include two projects of the harmonisation programme. Project Reviewed Population Vital and Census Statistics, Estonian counties (1995-2001) addresses harmonisation of regional population statistics, in its first stage for the period 1965-1990. Project Estonian Reviewed Population Birth Statistics (1995-2001) brings into scholarly circulation archived birth records, the project started with intercensal interval 1959-1970. However, the subtheme has been supported by several other projects which have included relevant methodological sections.

3.1.2.2. The principal international frameworks of the subtheme include cooperation with institutions responsible for methodological coordination in the field of population data, including European Population Observatory, UN Population Division, UN Statistical Division, UNECE, Eurostat, ILO Statistical Office, CIESIN, but also national statistical offices in other countries. The principal national cooperation has involved Statistical Office, National Archives, Institute of Experimental Medicine, county governments etc.

3.1.2.3. Research and activities united under the subtheme have generated a variety of new population information, used extensively for research as well as decision-making purposes, on national, sub-national and international level. Also, the subtheme has given rise to the development of population archive at the Institute, assembling various published and unpublished materials, instructions, correspondence etc.

3.1.2.4. Research and activities united under the subtheme have been a driving force to re-build the national statistical system. The initiative of the Institute lead to the establishment of Governmental Commission for Population and Social Statistics, bringing together about 40 government agencies and departments, participating in national statistical system, research community and high-ranking executives. The Institute has prepared reports on the development of national statistical system to different government institutions, including the Office of the President, Parliament commissions, Office of Prime Minister, Ministry of Finance, National Research and Development Council etc. The experience has been shared with the countries facing similar tasks and statistical context, in particular the Baltic and Caucasian region.

3.1.3. Major results

The major research under subtheme results can be summarised in the following:

• methodological principles for the rebuilding of national statistical system have been elaborated, synthesising the experience from the prewar period, international standards in the field and the experience of Baltoscandian and other European countries [3; 11; 12; 13; 16; 27; 33; 49; 50; 61]
• published and unpublished information on births, deaths and migrations has been retrieved, data quality evaluation and harmonisation performed for inter-censal age structures on national and county level (1959-1970, 1970-1979 and 1979-1989); harmonised age structures have provided a basis for complete revision of population-based indicators [53; 54]
• in cooperation with Statistical Office, a revised series of post-censal age structures has been estimated for the period 1989+; new age structures have confirmed significant deterioration in the accuracy of stock data during the 1990s, with distinct regional and age-pattern, and considerable effect on recent demographic trends [28; 64]
• data quality evaluation and analysis, using indirect techniques of estimation, has been launched to bridge the gap in age structures introduced by the war and repression years (intercensal period 1941-1959); the results for the 1950s have been tested through an application in French-Baltic comparative mortality research project [14]
• for the time in Estonia, a continuos set of annual national life tables (starting from 1922, with the gap in 1940-1949) has been prepared, applying time-consistent methodology and estimation procedures; the analysis revealed significant but more importantly varying across time discrepancies with intermittent "old" series; a set of sub-national life tables centred around 1989 census has been prepared, also for the first time in the country [14; 55; 56]
• regarding vital statistics, the methodology and procedures have been developed for the retrospective computerisation of birth records, introducing internationally comparable definitions and classifications, including time-consistent register of NUTS-equivalent territorial units, occupations etc [8; 66; 67]
• in cooperation with Civil Registration Office, systematic computerisation of archival records has been initiated; retrieval and harmonisation of birth records for intercensal period 1959-1970 has completed (over 200 thousand individual records), a full set of comparable tables has been prepared, covering county as well as community level (CD-ROM) [8]
• in-depth analysis of the 1989 census methodology has been accomplished, covering definitions, enumeration as well as coding procedures; the consistency of microdata has been verified, dataset documented, and where appropriate, internationally comparable definitions/classifications added; in cooperation with UNECE, the methodology has been applied for the census re-codification for the ex-USSR countries, including Russia [4; 15; 17; 43; 57; 59; 62]
• pilot study has been accomplished to retrieve from the archive individual records of the 1959 population census; the study revealed the completeness of records; relevant methodology and procedures were developed and applied to the computerisation census records for LÃÃnemaa county; the international importance of the work is underlined by the fact the only individual records left from the 1959, the first postwar census are those in Estonian archives
• in the framework of the cooperation with UNECE, a national database with complete coverage older population (50+) and their household members has been developed; the database is deposited at UNECE and available for international research community [41; 42; 43]
• vital and census statistics on county and community level (1965-1990) has been harmonised and introduced into scholarly and applied circulation; based on harmonised data, the analysis of regional population development has been accomplished for seven counties and in progress for others [1; 2; 5; 6; 7; 9; 10; 30; 31; 37; 38; 39]
• harmonised population data have been used for population projections, including regional, prepared in cooperation with local governments; the results of the projections have been used for a variety of purposes, ranging from elderly care to privatisation of major state/municipal-owned companies [52; 69; 71]
• the analysis of the quality of the registration of residential moves has been carried out; the results showed remarkable deterioration in the coverage of registration procedures since the end of 1980s, particularly for the age groups with the highest migration intensity, and the need for careful data reconstruction by means of indirect techniques [4; 29; 32]

3.2. Integrated survey statistics

3.2.1. Background

After the WWII, a major development with respect to population data sources has been the rapid progress of survey statistics. Supporting human-centred perspective which could not be available otherwise, sample surveys have become an indispensable supplement to vital registration and population census. In developed nations, the emergence and advancement of survey statistics has been closely related with population science.

Research community has contributed to this process directly, through methodological work and its applications, as well as indirectly, in the capacity of demanding professional users. No less importantly, on the other way around, modern population research is barely imaginable without the wealth and variety of information derived from nationally representative surveys.

In case of Estonia the referred linkage was virtually non-existent until the 1990s. Although a series of demographic surveys had been carried out in the Soviet Union starting from the 1960s, these surveys were grossly non-representative from the viewpoint of Estonian population. Similarly to population data in general, survey methodology was developed beyond the country and the release of individual-level data, resulting from such exercises, was legally prohibited to academic institutions.

On the other hand sociological surveywork, quite extensive in Estonia, suffered from the lack of quality-oriented statistical procedures. When the re-building of national statistical system became feasible in the early 1990s, the Institute, following the concept of NPI, took the initiative and became extensively involved in the process. Aside the methodological commitment, the Institute became closely involved in the development of environment infrastructure required for the implementation of methodologically demanding large-scale surveys.

Research and activities under the subtheme have included at least four complementary directions. First, the Institute has been engaged in the elaboration of methodology for surveys with substantive focus, covering major fields of population and societal development. Second, the Institute developed sampling frame and procedures to ensure representativeness, previously not available in Estonia. Third, through methodology, relevant analyses, training and supervision the Institute contributed to the establishment of interviewer network at Statistical Office. And fourth, the implementation of national surveys required the introduction of appropriate funding practices to secure the continuity across budget years. Due underdeveloped statistical environment, instead of applying the existing routines, these tasks turned out to be innovative, requiring extensive scholarly input.

The development of survey statistics in Estonia has progressed according to the plan formulated by the Institute in 1993 and adopted by Governmental Commission for Population and Social Statistics. As a special institutional arrangement, the Institute initiated working groups for major national surveys, uniting researchers irrespective of affiliation, administrators and foreign experts, and having full responsibility for all aspects of the survey, including the disposal of funding.

The arrangement proved to be particularly appropriate to the conditions of a small country, and the working group for Estonian Family and Fertility Survey has been operational for ten years already, and being reconstituted, focuses now on the preparation of FFSII/GGS. Integrated national surveys, resulting from research and activities under subtheme form an invaluable new source on Estonian population development.

3.2.2. Subtheme frameworks

3.2.2.1. The principal project frameworks of the sub-theme have been the Estonian Family and Fertility Survey, Estonian National Minority Survey, Estonian Labour Force Survey and Estonian Health Survey. The referred projects, covering the development of methodology, programme and survey instrument, sampling, field operations, coding, data entry, data cleaning and standard publication programme, have been operating under Governmental Commission for Population and Social Statistics / Ministry of Finance. Analytical projects, based on the referred surveys, are included under other themes.

3.2.2.2. The principal international cooperation frameworks of the subtheme have been the Informal Working Group of the European FFS at United Nations Economic Commission for Europe (1988+) and Network for Integrated European Population Studies, NIEPS (2000-2002). The principal national cooperation frameworks include working groups for Estonian Family and Fertility Survey, Estonian National Minority Survey, Estonian Labour Force Survey and Estonian Health Survey which assemble scientists from different national research institutions and experts from government institutions.

3.2.2.3. Research and activities under the subtheme have generated a set of five integrated event history surveys, based on comparable definitions and procedures. Data from the surveys has been used extensively for substantive analyses under theme 2.1, in particular under subthemes 2.1.2, 2.1.3 and 2.1.4.

3.2.2.4. Most importantly, the results of the subtheme are considered in the framework of FFS2IIGGS, leading European comparative research programme in the population field. Methodological experience and results gained at the Institute are especially relevant for FFSII/GGS since the programme, compared to first FFS round, emphasises stronger integration across priority areas population development. In particular, the FFSII/GGS aims to extend it focus to later phases of life careers and demographic, social and cultural heterogeneity of immigrant populations, which domains have been addressed by integrated surveys. Additionally, the experience gained in the field of survey statistics has been shared with countries with similar tasks and statistical context. On the national level, Statistical Office has been the key beneficiary of subtheme research and activities.

3.2.6. Major results

The major research results under the subtheme can be summarised in the following:

• the principles and programme for national survey statistics has been developed, coordinated with international organisations and implemented by relevant national institutions, including Statistical Office [18; 19; 33; 70]
• in cooperation with national working groups, the methodology for five national event history surveys has been developed; the methodology was adapted to Estonian societal and statistical context but maintaining international comparability [15; 18; 40; 45]
• the sample frame, covering the total population of the country, allowing for different sampling schemes and criteria, has been developed; particularly important has been the availability of extensive information (up to 55 characteristics) on respondents as well as non-respondents, enabling the quality-supporting procedures [24; 62]
• the analysis of representativeness, response and non-response by major population charateristics has been conducted for integrated national surveys; special analysis has been performed to evaluate the effect of contact procedures on survey participation [20; 51; 64]
• the concept for building up the governmental interviewer network, including the geographical coverage, professional requirements etc has been elaborated and delivered to Statistical Office; interviewer training for nationally representative surveys has been conducted, totaling over twenty seminars [21; 26; 35]
• the analysis of interviewer performance by general and survey-specific experience, language compatibility and other characteristics has been conducted; the results demonstrated significant data-quality impact which can be controlled by appropriate network management; the study was acknowledged by the Cochran-Hansen Award of International Association of Survey Statisticians [23; 25]
• the analysis of date-specific non-response has been performed for national event history surveys, the results showed relatively even distribution of recall problems across questionnaire modules, except the concentration in parental home module, the extension of cohort range to ages 70-80 did not involve major deterioration of data accuracy [24; 40; 62]
• special validity study, based on linkage with hospital records, was performed to evaluate the reliability of survey responses on sensitive topics, in particular the experience of induced abortion; despite certain variation among women, reporting rate of recent aborting proved to very high by international standards, thus supporting the development of reproductive health modules in the Estonian FFS and Estonian Health Survey [48; 60; 63]
• the linkage of survey responses to census records of the respondents was established; linked records have been used for the analysis of definitions of major population characteristics in census and survey statistics, the results identified several population categories who have been dissimilarly represented in two sources (e.g. cohabiting partners, persons employed in small-scale agricultural sector etc); record linkage has been also used for the estimation of recall bias in retrospective questions [40; 45; 47]
• standardised model for publishing the results of national surveys, covering methodology, numerical results and analysis, procedures for coordinating the planning and implementation of national surveys with government institutions, and in cooperation with National Archives, the standards for archiving has been developed; by the Institute, the full survey cycle has been completed for female and male FFS and National Minority Survey [22; 46; 47]
• participating in all working groups, the Institute has secured the methodological consistency of major definitions, classifications and procedures applied in different surveys, allowing their integration; following data quality evaluation and harmonisation, an integrated database (24 thousand individual records) has been developed [22; 40]

3.3. Estonian Population Databank

3.3.1. Background

The research and activities under the previous two subthemes have aside scientific results, generated new population-related information. In other words, the Institute has acted in the capacity of data producer as it is to varying extent characteristic to any NPI. In the Estonian context, however, the role of the activities is underlined by the fact that the Institute has been the leading data harmonising institution in the country which contribution goes beyond the scholarly use. To make the methodology and data available for the wider range of purposes, relevant information has been brought together in the general framework of Estonian Population Databank (ERA) and specific activities have been development for its dissemination.

The development of the databank is based on the output of the previous two subthemes. In describing the ERA, two aspects should be particularly underlined. Firstly, the ERA is not limited to computerised data on population indicators but covers also various methodological information concerning definitions, classifications, procedures etc. Correspondingly, the release of information from ERA does not mean so much the release of data but the data accompanied with appropriate methodological explanations. Secondly, different sections of the databank have not been developed in isolation but interact with each other. New input in one section usually leads to the upgrading in other sections, for example the introduction of adjustments in age structures, leads to recalculation all population-based indicators. Such links between sections have been considered already when planning various projects under the theme.

While the participation in methodology development for national statistical system belongs to the mission of NPI, the functions included in the thematic activity are usually not implemented by national population institutes but rather by statistical offices. In case of Estonia, the transfer of such responsibilities to the Institute stems from the discontinuity of national statistical system, discussed above. The tasks considered under thematic activity have required considerable inputs in time and resources, often with no refunding from appropriate sources. Nevertheless addressing these tasks has been a deliberate choice because it would have been otherwise impossible to secure appropriate database for research at the Institute, particularly in the field of survey statistics.

It should be noted that the results under thematic activity are not presented for scientific evaluation but rather for an overview of activities of the Institute in a broader societal context. Therefore, refernces for publications are note included, and if considered irrelevant, the following sections can be skipped.

3.3.2. Frameworks of thematic activity

3.3.2.1. During the past ten years the thematic activity has received the input from the previous two subthemes, population data harmonisation and integrated survey statistics. In addition, thematic activity has been supported by Governmental Commission for Population and Social Statistics.

3.3.2.2. National collaboration in data producing has included cooperation with Statistical Office, Medical Statistics Bureau and Medical Birth and Abortion Register. The main partner using the results of the thematic activity has been Statistical Office (Ministry of Finance), other users have the Department of Social and Economic Analysis at the Parliament, Institute for Experimental and Clinical Medicine, Institute of Estonian Language, Tartu University etc. Starting from 1994, the main international cooperation framework has been the European Population Committee at Council of Europe, coordinating the data requests with UN Statistical Office and Eurostat. Collaboration is also exercised with European Population Observatory, UN Population Division US Census Bureau, UNECE etc.

2.1.3. Major results

The major results under the thematic activity can be summarised in the following:

• the Institute has initiated the establishment of Governmental Commission for Population and Social Statistics (1993-1998), uniting scientists and high-ranking officials from relevant ministries; the aim of the Commission was to coordinate the rebuilding of national statistical system
• the Institute has served for a methodological coordination unit for national surveys; researchers of the Institute have participated in respective working groups in order to deliver relevant methodology and secure the consistency of major definitions, classifications and procedures; activities in the referred direction have created possibility to develop an integrated database
• the Institute has proposed the establishment of national interviewer network at Statistical Office; after the proposal was approved by the Governmental Commission for Population and Social Statistics, the Institute elaborated the principles for the network and participated in several waves of training seminars for the interviewers
• the Institute has been responsible for harmonisation programme of population data, under which, among others Statistical Office has been delivered harmonised age structures for intercensal intervals 1959-1970, 1970-1979, 1979-1989 and revised age structures for the period 1989+; the data has been used by Statistical Office for the recalculation of population-based indicators and dissemination to national as well as international users
• in cooperation with Goskomstat (and Institute of Statistics), Russian Federation and Estonian Statistical Office, based on the authorisation by the Ministry of Foreign Affairs, microdata of the 1979 and 1989 population censuses have been returned
• the methodology, procedures and microdata of the 1989 population census has been evaluated, and where appropriate, internationally comparable definitions and classifications added; with permission of Governmental Commission for Population and Social Statistics, individual census records have been shared with UNECE, being a major breakthrough after decades of data restrictions in Eastern Europe
• the Institute has initiated the participation of Estonia in the International Earth Science Information Network (CIESIN); based on available data, together with Poland, Estonia was selected a site for CIESIN pilot node in Central and Eastern Europe; under the coordination of the Institute, national steering committee was approved by the government, a country node was established at National Library to assemble metadata on major national databases, series of dissemination seminars were organised in cooperation with UNDP Headquarters
• to introduce population information into high-level decision-making process, cooperation with the Department of Social and Economic Analysis at the Parliament has been developed, among others, seminar has been conducted for parliamentarians and parliament councilors, explaining the principles of national statistical system and critical data quality issues
• in the framework of governmental programme, regional development plans at county level have been prepared under the coordination of the Ministry of Environment; in cooperation with UNDP and Stockholm Environment Institute (Tallinn branch), the Institute has introduced the methodology of development planning to county governments, also at the special seminar to senior planning officers
• in cooperation with the Ministry of Foreign Affairs and Council of Europe, the Institute has conducted a seminar targeted to diplomatic missions in Estonia, presenting the development of national minorities and immigrant in the country, based on internationally comparable concepts and definitions
• the Institute has taken several steps to coordinate and methodologically support the reorganisation of national population statistics comparatively in Estonia, Latvia and Lithuania; among others, executive seminar has been organised, under the aegis of Governmental Commission for Population and Social Statistics, in cooperation with UN Statistical Office, national statistical offices in Nordic countries and State Chancellor; the seminar resulted in general agreement on data collection programme and registration forms in vital statistics and publication as well as the programme of national surveys
• considering "common starting point" at the turn of 1990s, methodological experience accumulated in the process of rebuilding the national statistical system in Estonia has been shared with three Caucasian countries, Armenia, Azerbaijan and Georgia; the needs of population statistics and research in Caucasian countries have been brought to wider international consideration, including European Population Committee;
• the Institute has been asked to several expert missions by international organisations, among others Council of Europe/EU census mission to Macedonia and High Level Expert Group to FYROM, Eurostat mission to harmonise the labour force statistics in Bulgaria and World Bank mission to Georgia to prepare a national report on reproductive health

3.4. Publications

Publications 1996-2003

Publications 1991-1995

Electronic databases

1. Estonian Population databank
2. Estonian Fertility database 1926-1986. Intercensal Period 1959-1970
3. Estonian Fertility database 1926-1986. Intercensal Period 1970-1979
4. Estonian Family and Fertility Survey. Female database
5. Estonian Family and Fertility Survey. Male database
6. 1959 Estonian Population Census. Läänemaa county
7. 1985 Estonian Microcensus
8. Estonian Labour Force Survey database
9. Estonian National Minorities Survey database
10. Estonian Population Ageing database
11. Estonian Database on Population related policies