Figures and data in Evaluating the results of a social benefit simulation using individual administrative data on benefit receipt

Abstract

The simulation of social benefit entitlements is an important application of tax-benefit microsimulation models in social policy research. However, the simulation of social benefits that include a comprehensive means test and overlap with other social benefits is subject to numerous data limitations and simulation inaccuracies. This note examines the validity of the results of an open-source tax-benefit microsimulation model (GETTSIM) on social benefit entitlements. We use accurate administrative data collected during the application process for the simulation to assess the quality of the simulation. We find a low beta error rate and a high match between simulated and recorded benefits, making GETTSIM a powerful tool for the analysis of social policies.

This article contains 3 figures and 8 tables.

Figures

**Figure 1.** Unemployment Benefit II (UBII), Housing Benefit (HB) and Child Benefit (CB) for a single-earner couple household with one minor child (2023).

**Figure 2.** Distribution of the monthly simulated and recorded UBII entitlements.

**Figure 3.** Distribution of the individual deviation of the simulated monthly UBII entitlement from the recorded UBII entitlement (Euro).

Tables

Table 1. Sample selection.

	2017 Individuals	2017 Observations	2018 Individuals	2018 Observations
UBII recipients	152,266	1,492,013	145,126	1,432,790
⁒ recipients <25 or >64 years old	87,459	848,561	82,833	810,684
⁒ only children in the household registered	87,145	844,708	82,531	806,966
⁒ no valid simulation possible	79,124	730,964	74,672	696,506
= analysis sample	79,124	730,964	74,672	696,506

Source: SIG version 0720, own calculations.

Table 2. Data preparation for the simulation.

Input variable	Label	Source	Information used in SIG data, data transformation, assumption
hh_id	Household identifier	E	Number of the benefit unit
tu_id	Tax Unit identifier	D/A	Singles, single parents and couples with married head of household and partner: tu_id equals hh_id. Couples with at least one unmarried partner, the head of household and partner receive a separate tu_id and children in the household are randomly assigned to one of the two tu_ids. Calculation based on marital status and type of benefit unit.
p_id	Person identifier	E	Unique personal identifier
kind	Dummy: Dependent child living with parents	E	Position in the benefit unit
bruttolohn_m	Monthly gross wage	E	Monthly gross wage considered in the means test
alter	Individual’s age.	E	Age
weiblich	Dummy: female	E	Gender
rentner	Dummy: Pensioner employment status	E	Reason for exclusion from SGB II
alleinerz	Dummy: Single parent	E	Type of benefit unit
wohnort_ost	Dummy: Living in former East Germany	E	Place of residence
in_priv_krankenv	Dummy: In private health insurance	A	Assumption that there is no private health insurance.
priv_rentenv_beitr_m	Monthly private pension contribution	A	Set to 0, assumption that there is no private pension insurance.
in_ausbildung	Dummy: Employment status “in education”	D	In vocational training or eligible for training assistance.
selbstständig	Dummy: Self-employed (main profession)	D	Set to 1 if income from self-employment is considered in the means test
hat_kinder	Dummy: Has kids (incl. not in hh)	D/A	Type of benefit unit, assumption: no children outside the household
betreuungskost_m	Monthly childcare expenses for a particular child under the age of 14	A	Set to 0
sonstig_eink_m	Additional income: includes private and public transfers that are not yet implemented in GETTSIM (e.g., BAföG, Kriegsopferfürsorge)	E	Other income considered in the means test (public and private transfers)
eink_selbst_m	Monthly income from self-employment	E	Monthly gross income from self-employment considered in the means test
eink_vermietung_m	Monthly rental income net of deductions	E	Monthly disposable rental income considered in the means test
kapitaleink_brutto_m	Monthly capital income	E	Monthly disposable capital income considered in the means test
bruttokaltmiete_m_hh	Monthly rent expenses for household	E	Accepted regular monthly gross rent including additional costs (excluding heating costs)
heizkosten_m_hh	Monthly heating expenses for household	E	Accepted regular monthly heating costs
wohnfläche_hh	Size of household dwelling	A	45 sqm for first person plus 15 sqm for each additional member
bewohnt_eigentum_hh	Dummy: Owner-occupied housing	A	No home ownership
arbeitsstunden_w	Weekly working hours of individual	A	Set to 0
geburtstag	Day of birth	E	Day of birth
geburtsmonat	Month of birth	E	Month of birth
geburtsjahr	Year of birth	E	Year of birth
jahr_renteneintr	Year of retirement	D/A	Year of birth +65
m_elterngeld	Number of months hh received elterngeld	A	Set to 0
m_elterngeld_vat_hh	Number of months father received elterngeld	A	Set to 0
m_elterngeld_mut_hh	Number of months mother received elterngeld	A	Set to 0
behinderungsgrad	Handicap degree (between 0 and 100)	A	Set to 50 if person is severely disabled, else set to 0
schwerbeh_g	Severely handicapped, with flag “G”	E	Set to 1 if person is severely disabled
mietstufe	Level of rents in city (1: low, 3: average)	E/A	Merged to dataset based on the municipality number of the place of residence, missing values are assigned to level 3
immobilie_baujahr_hh	Construction year of dwelling	A	Set to 2000
vermögen_bedürft_hh	Assets considered for means testing	A	Set to 0
entgeltp	Earnings points for pension claim	A	Set to 0
priv_rente_m	Amount of monthly private pension	A	Set to 0
bürgerg_bezug_vorj	Received Bürgergeld in previous year	A	Set to 1
kind_unterh_anspr_m	Monthly gross child alimony payments as determined by the court	D	Maintenance payments (for children and a spouses) considered in the means test
kind_unterh_erhalt_m	Monthly actual child alimony payments	D	Maintenance payments (for children and a spouses) considered in the means test
steuerklasse	Tax Bracket (1 to 5) for withholding tax
jahr		E	Year of recorded benefit receipt
Specified output variables
unterhaltsvors_m	Alimony advance payment	E/A	Set to 0 (included in kind_unterh_erhalt_m)
veink_algI	Monthly amount of unemployment insurance benefit	E	Unemployment insurance benefit considered in the means test
ges_rente_m	Total private and public pension	E	Pension payments considered in the means test (public and private)

Source for the variable name and description: https://gettsim.readthedocs.io/en/stable/gettsim_objects/input_variables.html [accessed September 23, 2024]. E=Existing variable used or transformed, D=New variable derived/approximated from available information, A=New variable generated by assumption. Specified output variables are variables that can be simulated, but were specified as input variables in our analysis and hence not simulated.

Table 3. Simulated versus registered UBII entitlement (shares in percent).

	GETTSIM	NoRentLimit	HAStricter1	HAStricter2
All observations	100.00	100.00	100.00	100.00
with simulated UBII entitlement	90.66	91.05	96.32	97.12
without simulated UB II entitlement (beta error)	9.34	8.95	3.68	2.88
thereof:
simulated housing and child benefit	72.95	74.64	18.05	0.01
simulated housing benefit only	23.17	21.72	23.80	4.26
simulated child benefit only	0.61	0.59	44.03	68.68
none	3.27	3.04	14.12	27.05

Notes: Beta-error results refer to 1,427 thousand person-month-observations that enter the simulation in each simulation variant.

Source: SIG version 0720, GETTSIM v0.7.0, own calculations.

Table 4. Mean, standard deviation and quantiles of the distribution of recorded and simulated UBII.

	Recorded UBII-entitlement	GETTSIM	NoRentLimit	HAStricter1	HAStricter2
Mean	891	873	882	865	859
Std. deviation	500	417	427	427	430
0.1 quantile	390	413	415	409	404
0.5 quantile	796	794	796	786	784
0.9 quantile	1,515	1,456	1,465	1,447	1,444
N (in thousands)	1,427	1,294	1,300	1,375	1,386

Source: SIG version 0720, GETTSIM v0.7.0, own calculations.

Table 5. Distribution of UBII single payments, payments for special needs, contributions to health insurance and sanctions (Euro per month).

	Single payments housing costs	Payments for special needs	Contributions to health insurance	Sanctions
Mean	5	18	177	8
Std. deviation	90	43	90	65
0.1 quantile	0	0	0	0
0.5 quantile	0	0	117	0
0.9 quantile	0	60	250	0
0.99 quantile	0	163	470	240

Source: SIG version 0720, own calculations.

Table 6. Recorded and simulated UBII entitlements by subgroup (mean values (standard deviation) in Euro per month, simulated mean value (standard deviation) relative to recorded mean value (standard deviation) in percent).

	Recorded UBII-entitlement	GETTSIM		NoRentLimit		HAStricter1		HAStricter2
Females	912	907	99%	917	101%	890	98%	883	97%
Females	(512)	(431)	(84%)	(440)	(86%)	(442)	(86%)	(446	(87%)
Males	871	841	97%	850	98%	841	97%	836	96%
Males	(487)	(400)	(82%)	(412)	(85%)	(410)	(84%)	(412)	(85%)
Singles	655	648	99%	657	100%	654	100%	650	99%
Singles	(231)	(189)	(82%)	(199)	(86%)	(202)	(87%)	(206)	(89%)
Single Parents	885	905	102%	913	103%	851	96%	844	95%
Single Parents	(399)	(292)	(73%)	(301)	(75%)	(337)	(84%)	(343)	(86%)
Couple w/o children	880	873	99%	879	100%	875	99%	873	99%
Couple w/o children	(389)	(354)	(91%)	(358)	(92%)	(361)	(93%)	(363)	(93%)
Couple w/ children	1,262	1,258	100%	1,268	100%	1,170	93%	1,158	92%
Couple w/ children	(617)	(420)	(68%)	(439)	(71%)	(472)	(77%)	(480)	(78%)
25-<35 years old	905	843	93%	852	94%	827	91%	821	91%
25-<35 years old	(510)	(374)	(73%)	(386)	(76%)	(386)	(76%)	(390)	(76%)
35-<45 years old	999	995	100%	1004	101%	973	97%	966	97%
35-<45 years old	(572)	(471	(82%)	(481)	(84%)	(481)	(84%)	(485)	(85%)
45-<55 years old	869	880	101%	891	102%	881	101%	876	101%
45-<55 years old	(475)	(436)	(92%)	(446)	(94%)	(447)	(94%)	(449)	(95%)
>54 years old	755	759	100%	768	102%	761	101%	756	100%
>54 years old	(357)	(330)	(92%)	(338)	(95%)	(341)	(96%)	(344)	(96%)
With income from self-employment	830	896	108%	905	109%	879	106%	873	105%
With income from self-employment	(430)	(420)	(98%)	(423)	(98%)	(420)	(98%)	(424)	(98%)

Note: Grey cells represent the deviation from the group mean of the registered UBII claims, with increasing deviations for darker colours.

Source: SIG version 0720, GETTSIM v0.7.0, own calculations.

Table 7. Regression results.

Explanatory variables	Dependent variable: Deviation of monthly recorded UBII-entitlement (in euros) from simulated entitlement according to simulation variant...
Explanatory variables	NoRentLimit			HAStricter2
	Coefficient	Stat. Sig.	N	Coefficient	Stat. Sig.	N
Age (years)	-10.45	***	1169	-13.44	***	1269
Age_squared (years)	0.11	***	1169	0.14	***	1269
Female (ref.: male)	-1.08	***	548	-3.08	***	607
Marital status (ref.: single)
Married, living separately	7.8	***	55	12.98	***	60
Married, living together	-12.04	***	354	-12.65	***	403
Divorced	11.63	***	120	16.15	***	128
Widowed	-6.91	***	13	-5.44	***	15
Unknown	-38.65	*	0	-61.85	*	0
Employment status (ref.: unemployed)
Regular employed	-16.23	***	134	-29.71	***	168
Vocational training	-15.65	***	9	-31.49	***	11
Marginal employment	-17.55	***	135	-28.17	***	141
Else	57.94	***	5	51.22	***	5
Country/region of nationality (ref.: Germany)
EU	-10.54	***	84	-7.36	***	95
Turkey	-14.04	***	58	-9.56	***	65
Balkan countries	1.33		23	5.78	***	26
Eastern Europe	4.35	***	16	6.75	***	17
Africa	26.71	***	26	38.21	***	29
America	-5.81	**	4	9.89	**	4
Asia	32.29	***	151	61.24	***	168
Else/Unknown	12.36	***	4	98.16	***	5
Status when entering Germany (ref.: No status, no information, missing)
(ethnic German) Resettler (Spätaussiedler)	-9.52	***	3	-14.36	***	3
Person entitled to asylum	3.63		4	14.07	***	4
Asylum seeker	26.43	***	4	26.11	***	4
Quota system refugee (Kontigentflüchtling)	-7.05	**	2	-16.52	***	2
Residence status (ref.: German, EU-resident, missing)
Permit of residence	6.13	***	176	11.72	***	198
Temporary permission to stay	59.14	***	6	115.76	***	7
Settlement permit	-19.42	***	82	-29.78	***	91
Visa	11.63	***	3	53.89	***	3
Other permission	-11.37	***	7	-15.17	***	8
Asylum application not yet submitted	67.87	***	2	78.78	***	2
Living space (square meters)	-0.23	***	1169	-0.33	***	1269
Number of people living in the household	25.71	***	1169	39.71	***	1269
Family type (ref.: Single)
Single parent	-5.51	***	153	-8.61	***	177
Couple w/o children	2.06	***	133	-4.99	***	136
Couple w/ children	59.95	***	240	49.16	***	293
Additional needs due to disability	121.96	***	3	116.96	***	3
Severe disability (ref.: No)
Yes	3	***	53	2.86	***	56
No information/missing	29.81	***	11	42.43	***	12
Eastern Germany (ref.: Western Germany)	3.76	***	328	9.36	***	355
Year 2018 (ref.: 2017)	-2.52	***	570	-1.22	***	619
Intercept	215.14	***		278.05	***
N (in thousands)	1,169			1,269
Adjusted R-squared	0.13			0.11
Outcome (mean, euros)	31			54

Notes: Asterisks */**/*** indicate statistical significance at the 10/5/1 per cent level, based on heteroscedasticity robust standard errors.

Source: SIG version 0720, GETTSIM v0.7.0, own calculations.

Table A1. Composition of the simulation sample and the excluded values where no simulation was possible (last sample selection step).

	Simulation Sample	No valid simulation possible	Difference
	years (mean)		absolute	p-value*
Age	42.6	43.8	-1.14	0.00
	shares
Female	0.49	0.40	0.09	0.00
Single	0.47	0.01	0.47	0.00
Single Parent	0.15	0.42	-0.27	0.00
Couple w/o children	0.10	0.02	0.09	0.00
Couple w/ children	0.27	0.52	-0.24	0.00
Eastern Germany	0.28	0.25	0.03	0.01
No school leaving certificate	0.21	0.18	0.03	0.00
No German citizenship	0.34	0.34	0.00	0.04
Observations	1,427,470	224,204

Source: SIG version 0720, own calculations.

p-value of a t test on the equality of means.