Survey wave export, German locale Passed

input 01_survey_export_de.csv · profile 01_survey.yaml (survey-de) · mode in-memory · clean-data 0.1.0

Read

1,026

Out

982

Rejected

Duplicates

Cells changed

5,333

Columns

Row flow

read 1,026 = out 982 + rejected 18 + duplicates 18 + empty 8. Reject rate 1.8 % against a gate of 5.0 %.

kept 982
rejected by validation 18
duplicates 18
empty records 8

Warnings

No warnings.

Notes:

structure.empty_columns_dropped empty columns dropped: column_14
structure.repeated_headers repeated headers numbered: kommentar_2

Header mapping

#	Raw	Normalised	Final	Action
1	Respondent ID	`respondent_id`	`respondent_id`	kept
2	Welle	`welle`	`wave`	renamed
3	Startdatum	`startdatum`	`start_date`	renamed
4	Abgeschlossen am	`abgeschlossen_am`	`completed_at`	renamed
5	Alter	`alter`	`age`	renamed
6	Geschlecht	`geschlecht`	`gender`	renamed
7	Land	`land`	`country`	renamed
8	Haushaltseinkommen (€)	`haushaltseinkommen_eur`	`household_income`	renamed
9	Ausgaben letzter Monat (€)	`ausgaben_letzter_monat_eur`	`spend_last_month_eur`	renamed
10	Zufriedenheit	`zufriedenheit`	`satisfaction`	renamed
11	NPS	`nps`	`nps`	kept
12	Kommentar	`kommentar`	`comment`	renamed
13	Kommentar	`kommentar_2`	`interviewer_note`	renamed
14		`column_14`		dropped empty

Columns before and after

Column	Type	Non-empty before	Nulls after	Distinct after	Invalid	Filled	Changed	Details
`respondent_id`	string	1,017	0	982	0	0	5	details before: whitespace 3 · non-NFC 0 · control 0 · null-like 0 · length 8 to 11 top before: `"NG-129914"` ×2, `"NG-189048"` ×2, `"NG-272155"` ×2, `"NG-283425"` ×2, `"NG-448991"` ×2 after: min – · max – · mean – · median – top after: `"NG-100388"` ×1, `"NG-103838"` ×1, `"NG-104803"` ×1, `"NG-105420"` ×1, `"NG-108657"` ×1
`wave`	integer	1,018	0	4	0	0	0	details before: whitespace 0 · non-NFC 0 · control 0 · null-like 0 · length 1 to 1 top before: `"4"` ×303, `"2"` ×239, `"3"` ×239, `"1"` ×237 after: min 1 · max 4 · mean 2.59165 · median 3
`start_date`	date	1,018	0	479	10	0	727	details before: whitespace 0 · non-NFC 0 · control 0 · null-like 0 · length 5 to 10 top before: `"16.05.2025"` ×5, `"06/02/2025"` ×4, `"08/08/2026"` ×4, `"12.01.2025"` ×4, `"12.08.2026"` ×4 after: min 2025-01-07 · max 2026-08-27 · mean – · median – patterns: `%Y-%m-%d` 257, `%d.%m.%Y` 529, `%d/%m/%Y` 222
`completed_at`	datetime	1,018	0	982	0	0	982	details before: whitespace 0 · non-NFC 0 · control 0 · null-like 0 · length 16 to 16 top before: `"02.06.2025 14:49"` ×2, `"10.12.2025 15:04"` ×2, `"11.01.2026 12:08"` ×2, `"12.11.2025 07:07"` ×2, `"13.03.2026 10:04"` ×2 after: min 2025-01-08T14:54:00 · max 2026-08-29T15:18:00 · mean – · median – patterns: `%d.%m.%Y %H:%M` 1,018
`age`	integer	1,018	3	68	0	0	3	details before: whitespace 0 · non-NFC 0 · control 0 · null-like 3 · length 2 to 4 top before: `"63"` ×24, `"34"` ×23, `"30"` ×21, `"55"` ×20, `"75"` ×20 after: min 18 · max 85 · mean 51.3841 · median 52
`gender`	string	1,018	0	4	0	0	982	details before: whitespace 1 · non-NFC 0 · control 0 · null-like 0 · length 1 to 12 top before: `"W"` ×167, `"keine Angabe"` ×152, `"w"` ×146, `"weiblich"` ×146, `"divers"` ×139 after: min – · max – · mean – · median – top after: `"female"` ×446, `"male"` ×257, `"not_stated"` ×145, `"diverse"` ×134
`country`	string	1,018	0	3	0	0	647	details before: whitespace 0 · non-NFC 0 · control 0 · null-like 0 · length 2 to 11 top before: `"AT"` ×182, `"de"` ×179, `"\u00d6sterreich"` ×177, `"DE"` ×165, `"Deutschland"` ×159 after: min – · max – · mean – · median – top after: `"DE"` ×484, `"AT"` ×346, `"CH"` ×152
`household_income`	number	998	0	203	0	20	982	details before: whitespace 0 · non-NFC 0 · control 0 · null-like 0 · length 6 to 10 top before: `"41.500,00"` ×9, `"63.500,00"` ×8, `"42.500"` ×7, `"45.000,00"` ×7, `"49.500,00"` ×7 after: min 18,000 · max 119,500 · mean 68,561.1 · median 68,000 missing: median with 68000.0, 20 filled
`spend_last_month_eur`	number	1,018	0	982	0	0	982	details before: whitespace 1018 · non-NFC 0 · control 0 · null-like 0 · length 7 to 10 top before: `"1.019,32\u00a0\u20ac"` ×2, `"1.187,14\u00a0\u20ac"` ×2, `"1.193,33\u00a0\u20ac"` ×2, `"1.434,12\u00a0\u20ac"` ×2, `"1.523,82\u00a0\u20ac"` ×2 after: min 20.41 · max 2,499.31 · mean 1,241.24 · median 1,230.76
`satisfaction`	integer	1,003	0	10	0	15	15	details before: whitespace 0 · non-NFC 0 · control 0 · null-like 0 · length 1 to 2 top before: `"7"` ×116, `"8"` ×113, `"1"` ×107, `"3"` ×107, `"10"` ×105 after: min 1 · max 10 · mean 5.60285 · median 6 missing: median with 6, 15 filled
`nps`	integer	1,018	0	11	0	0	0	details before: whitespace 0 · non-NFC 0 · control 0 · null-like 0 · length 1 to 2 top before: `"5"` ×112, `"0"` ×103, `"8"` ×103, `"1"` ×101, `"7"` ×99 after: min 0 · max 10 · mean 4.7556 · median 5
`comment`	string	588	422	8	0	0	5	details before: whitespace 2 · non-NFC 1 · control 5 · null-like 0 · length 11 to 31 top before: `"Lieferung versp\u00e4tet, sonst gut"` ×98, `"Sehr zufrieden mit dem Service"` ×83, `"Gerne wieder"` ×73, `"Keine Anmerkungen"` ×68, `"Mehr Auswahl w\u00e4re sch\u00f6n"` ×68 after: min – · max – · mean – · median – top after: `"Lieferung versp\u00e4tet, sonst gut"` ×86, `"Sehr zufrieden mit dem Service"` ×82, `"Gerne wieder"` ×72, `"Preis und Leistung stimmen"` ×67, `"Mehr Auswahl w\u00e4re sch\u00f6n"` ×66
`interviewer_note`	string	253	737	6	0	0	3	details before: whitespace 89 · non-NFC 0 · control 3 · null-like 0 · length 18 to 43 top before: `"Teilnahme per Tablet"` ×86, `"Telefon besetzt, zweiter Versuch"` ×86, `"R\u00fcckruf vereinbart"` ×78, `"Interview unterbrochen\r\nam Folgetag beendet"` ×1, `"R\u00fcckruf vereinbart\r\nabends erreichbar"` ×1 after: min – · max – · mean – · median – top after: `"Telefon besetzt, zweiter Versuch"` ×84, `"Teilnahme per Tablet"` ×83, `"R\u00fcckruf vereinbart"` ×75, `"Interview unterbrochen\nam Folgetag beendet"` ×1, `"R\u00fcckruf vereinbart\nabends erreichbar"` ×1

Rules

Rule	Checked	Failed	Examples
`respondent_id.required`	1,018	1	r952 ""
`respondent_id.regex`	1,017	2	r174 "NG-6346397" r855 "NG-17261"
`wave.type`	1,018	0
`wave.required`	1,018	0
`wave.allowed`	1,018	0
`start_date.type`	1,018	10	r66 "32.01.2025" r233 "31.02.2025" r480 "2025-00-10" r578 "2025-13-01" r711 "15/13/2025"
`start_date.required`	1,018	0
`completed_at.type`	1,018	0
`completed_at.required`	1,018	0
`age.type`	1,015	0
`age.min`	1,015	2	r48 "15" r76 "12"
`age.max`	1,015	3	r118 "130" r544 "150" r871 "101"
`gender.allowed`	1,018	0
`country.allowed`	1,018	0
`household_income.type`	998	0
`household_income.min`	998	0
`spend_last_month_eur.type`	1,018	0
`spend_last_month_eur.min`	1,018	0
`satisfaction.type`	1,003	0
`satisfaction.min`	1,003	0
`satisfaction.max`	1,003	0
`nps.type`	1,018	0
`nps.min`	1,018	0
`nps.max`	1,018	0
`dedupe.exact`	1,000	12	r85 duplicate of r80 r304 duplicate of r124 r324 duplicate of r181 r389 duplicate of r41 r407 duplicate of r189
`dedupe.key.1`	988	6	r40 superseded by r902 r190 superseded by r288 r220 superseded by r687 r422 superseded by r610 r528 superseded by r783

Duplicates

exact: 12 removed of 1,000 rows
dedupe.key.1 [respondent_id] keep last by completed_at: 6 removed

Rejects

36 rows in rejects.csv: read 0 · validate 18 · dedupe 18 · missing 0.

First failing rule	Rows
`age.max`	3
`age.min`	2
`dedupe.exact`	12
`dedupe.key.1`	6
`respondent_id.regex`	2
`respondent_id.required`	1
`start_date.type`	10

By column: age 5, respondent_id 3, start_date 10

First 20 rejected rows

Row	Stage	Rule	Detail	respondent_id	wave	start_date	completed_at	age	gender	country	household_income	spend_last_month_eur	satisfaction	nps	comment	interviewer_note
40	dedupe	`dedupe.key.1`	duplicate of r902	NG-129914	3	07.02.2026	07.02.2026 17:26	85	w	AT	79.500,00	1.825,82 €	2	5
48	validate	`age.min`	age.min: 15 < 16	NG-160720	1	11.04.2026	13.04.2026 10:55	15	m	AT	71.000	2.322,10 €	8	10	Keine Anmerkungen
66	validate	`start_date.type`	start_date.type: "32.01.2025" not a date (%d.%m.%Y, %Y-%m-%d, %d/%m/%Y)	NG-426903	1	32.01.2025	21.08.2025 09:02	48	keine Angabe	de	41.500	562,09 €	1	7	Sehr zufrieden mit dem Service	Teilnahme per Tablet
76	validate	`age.min`	age.min: 12 < 16	NG-679587	3	22.12.2025	22.12.2025 17:36	12	männlich	Österreich	81.500	2.489,90 €	8	0	Alles prima
85	dedupe	`dedupe.exact`	duplicate of r80	NG-928448	3	12.11.2025	12.11.2025 07:07	63	weiblich	Schweiz	49.500,00	1.187,14 €	10	1	Lieferung verspätet, sonst gut	Rückruf vereinbart
118	validate	`age.max`	age.max: 130 > 99	NG-774006	4	12/02/2026	13.02.2026 14:37	130	divers	de	48.000	1.413,39 €	1	7
174	validate	`respondent_id.regex`	respondent_id.regex: "NG-6346397" does not match ^NG-\d{6}$	NG-6346397	1	07/11/2025	07.11.2025 10:34	63	weiblich	AT	89.500	557,51 €	3	4	Mehr Auswahl wäre schön
190	dedupe	`dedupe.key.1`	duplicate of r288	NG-283425	4	11.02.2026	11.02.2026 21:14	56	divers	Deutschland	108.500,00	736,04 €	10	1	Lieferung verspätet, sonst gut	Telefon besetzt, zweiter Versuch
220	dedupe	`dedupe.key.1`	duplicate of r687	NG-798099	3	20.06.2025	20.06.2025 14:19	73	keine Angabe	AT	43.500	1.742,42 €	1	8
233	validate	`start_date.type`	start_date.type: "31.02.2025" not a date (%d.%m.%Y, %Y-%m-%d, %d/%m/%Y)	NG-679621	2	31.02.2025	11.11.2025 20:46	37	männlich	Österreich	60.000	2.042,77 €	7	9	Fragebogen war zu lang
304	dedupe	`dedupe.exact`	duplicate of r124	NG-272155	4	17.02.2026	19.02.2026 17:41	85	w	Österreich	47.000,00	1.603,66 €	2	7	Lieferung verspätet, sonst gut
324	dedupe	`dedupe.exact`	duplicate of r181	NG-761426	2	16.04.2025	16.04.2025 15:13	30	weiblich	DE	51.500	83,23 €	4	8	Lieferung verspätet, sonst gut
389	dedupe	`dedupe.exact`	duplicate of r41	NG-547873	3	10.01.2026	11.01.2026 12:08	84	keine Angabe	DE	54.000	781,01 €	10	0	Lieferung verspätet, sonst gut
407	dedupe	`dedupe.exact`	duplicate of r189	NG-801086	2	10.12.2025	10.12.2025 15:04	67	W	Deutschland	106.000	1.561,15 €	1	3	Lieferung verspätet, sonst gut
422	dedupe	`dedupe.key.1`	duplicate of r610	NG-959125	1	02.01.2026	03.01.2026 12:42	25	m	de	106.500	311,88 €	5	4	Gerne wieder	Rückruf vereinbart
480	validate	`start_date.type`	start_date.type: "2025-00-10" not a date (%d.%m.%Y, %Y-%m-%d, %d/%m/%Y)	NG-520961	4	2025-00-10	11.01.2025 08:18	74	w	DE	80.000	2.466,43 €	2	9	Keine Anmerkungen
528	dedupe	`dedupe.key.1`	duplicate of r783	NG-960378	4	06/07/2025	06.07.2025 21:23	45	weiblich	AT	79.000,00	1.523,82 €	10	7
544	validate	`age.max`	age.max: 150 > 99	NG-762228	2	07.09.2025	08.09.2025 11:21	150	keine Angabe	de	53.000,00	1.930,34 €	5	6	Gerne wieder
578	validate	`start_date.type`	start_date.type: "2025-13-01" not a date (%d.%m.%Y, %Y-%m-%d, %d/%m/%Y)	NG-920527	1	2025-13-01	11.07.2025 20:26	34	männlich	Schweiz	75.000,00	1.778,66 €	7	7	Alles prima
583	dedupe	`dedupe.key.1`	duplicate of r673	NG-649809	2	26.04.2025	26.04.2025 09:34	41	W	Deutschland	98.000,00	1.958,11 €	5	8	Lieferung verspätet, sonst gut	Telefon besetzt, zweiter Versuch

Run info

Started	2026-10-29T00:00:00Z
Status	ok · exit 0
Profile SHA-256	d8c56a41c66f1dfe1973ef0afb2c94f94c11039c13963f3e7a5aa2f62a131ad8
Input SHA-256	6ca4696f292c4a49139e0ab854d634fdfdaa2d863b74b2e4a6741efac23a0b79
Versions	clean-data 0.1.0, Python 3.13, pandas 3.1.0, pydantic 2.14.1, click 8.5.0, openpyxl 3.1.5, charset-normalizer 3.5.2, PyYAML 6.0.3, Jinja2 3.1.6, rapidfuzz 3.14.6, pyarrow 26.0.0