Overview, coverage and background
One database, built and owned by Oscar, developed around the UK public sector since 1984.
1
Database
423
Tables
51M
Rows
360M
Data points
1984
The database began with public sector knowledge, not web scraping.
Oscar was started by two Local Government Officers and has developed from that original research-led foundation into the connected public sector data environment used across the business today.
Started in 1984 by two Local Government Officers, our database remains the cornerstone of our business.
The brief is for the Oscar database to record all “Public Service Organisations” — all public and statutory bodies, as well as those providing services to the public that are partly or fully funded, directly or indirectly, by Government.
This backbone of organisational coverage gives us the platform to link a host of proprietary and open contact, structural, postholder, demographic and metric data points. Those feed a range of data-driven products and services, used across Oscar data, intelligence, research, engagement and delivery services.
Curating the database represents a significant annual investment across staff and systems. We have four times more research than sales staff — because nothing we do matters more than making sure the data is complete and accurate.
We have been developing our proprietary live database management software for 25 years, hosted on our own UK servers and built from the ground up around our unique taxonomies.
Research team
Fifteen UK-based researchers in four specialist teams, with an almost uniquely low level of staff change.
UK
Based
300 yrs
Collective experience
4
Specialist teams
Our internal research team comprises 15 UK-based researchers. Four department heads manage teams split across different organisation types, with each member a specialist in their particular area of the database.
We employ a full-time Head of Data Development and Bespoke Research who works across all datasets, monitors industry changes, establishes new taxonomies and sources new data points.
Many of the team have public sector backgrounds and represent over 300 years of collective experience. The average time with Oscar is over 10 years — one person has moved on in the last decade.
It typically takes three years of training and experience for a researcher to reach ‘admin’ level and make live updates without moderation — a level now reached by all but our two most recent appointments.
Oscar is not a scraped contact database.
Technology and AI can help our researchers identify change, compare sources and work more efficiently, but people remain responsible for validating, structuring and maintaining what enters the database.
Tech and software
A single live nexus for all data: one database on our own servers, with bespoke cloud software around it.
25 yrs
Of development
UK
Hosted
100%
Proprietary systems
We have been fans of cloud-based systems since the nineties. We moved to fully integrated online data management, storage and delivery in 2006.
The brief was simple: create a single live nexus for all data. We achieved this by hosting one database on our own web servers and building live, cloud-based custom software around it to manage every incoming and outgoing dataflow.
Incoming
- Research admin interface — management, scheduling, reporting and updating
- Customer admin interface — subscription, product access and CRM
- Websites — data subject and customer updates
- Data feeds — scheduled open, regulatory and other datasets
Outgoing
- Websites — display and interrogation
- Direct delivery — downloads and live feeds
Our stack
- Platform: Linux, Apache, MySQL, PHP, SQL, Python
- Frontend: JavaScript, AJAX, HTML, CSS, Bootstrap, jQuery
- Delivery: REST API, live feeds and scheduled downloads
Maintenance and updating
Three pillars: direct research by our own team, monthly liaison with data subjects, and open datasets for benchmarking.
30,000
Research hours a year
140,000
Monthly emails
82
Automated daily checks
We have built our own database health check and scheduling software to monitor record age, updates per team member, future events and erroneous data points — which our research heads use to trigger updates daily.
A monthly newsletter is sent to all postholders. Any undeliverable emails, or responses that mention changes, are passed to the research team for revalidation and update.
- 90,000 public service organisations
- 120,000 organisational sites including HQs
- Organisation-type taxonomies and sector classifications
- Full hierarchy: parent, child and sibling relationships
The starting point for each organisation type is as close to total coverage as possible, achieved through postal, telephone and email requests directly to organisations confirming name, main address, phone, email and — in larger bodies — annual spend and employee numbers.
We record a wide range of official ID numbers against our own Oscar number, then use automated processes to benchmark our entity data against published Government open and regulatory datasets. Differences are passed to the relevant research team to check directly.
Frequency: continuous change monitoring and scheduled checks, with a full research cycle across organisational coverage approximately every six months.
- 200,000 posts
- 700 functional categories
- 9 management levels
Postholder data is always updated through direct contact with, or information published by, the employing organisation. No third-party sources are used to source, validate or update postholder data.
Larger organisations (100+ staff): we liaise with an extensive network of nominated research contacts via their preferred method — email, post, telephone or published material. Smaller organisations: primarily telephone research.
At the same time we review functional responsibilities, management level and reporting structure, linking our standardised functional categories to each post. Those relationships drive our dynamic organograms and let the team sense-check the research visually.
Frequency: continuous change handling alongside a rolling full research cycle through postholder data approximately every six months, yielding an average record age of three months.
Schools
Head Teacher changes cluster around term start dates, so we flag and batch those updates. We also consume the monthly Government feed of educational establishments to monitor closures, new openings, academy conversions, MAT membership and pupil numbers.
Elected representatives
National election changes are applied on the night and the following day. Councillor changes and re-elections are recorded by the end of May, followed by roughly two months building out leadership, cabinet and committee structures. We also track boundary changes and by-elections.
Care homes
We pull the CQC location and provider file quarterly and run six daily automated checks on coverage, location, providers and bed numbers, flagging differences for the research team.
Housing associations
Annual Housing Regulator data on stock volumes, PRP group structures and housing types is compared and benchmarked against our own coverage before managed updates are applied.
Tenders and contract awards
Published tender requests and contract awards are collected daily and matched to the Oscar universe of buyers and our standardised supplier database, with unmatched names passed to researchers for manual linkage.
Geo data points
We consume ONS, NHS and specialist postcode and geo datasets, running 17 geo checks weekly to keep constituencies, devolved constituencies and regions, wards and electoral divisions correctly linked to their representatives.
Statutory posts and anomalies
- Weekly coverage checks on Chief Executives, s151 finance posts, Monitoring Officers and Public Health Directors
- Email and name formatting anomalies
- Organisations with deleted linked parents, or no main address/HQ
- Posts with missing functions or deleted parents
Unique standardisation and categorisation
Consistent IDs and categories are what make data usable across every part of public services.
Unique
Oscar IDs
800
Org types
700
Functional categories
9
Seniority levels
To make data usable when combining all organisation and post types from across public services, it is necessary to develop and use consistent categories and values. So we built our own.
Oscar ORGURNs and CONTACTURNs
Every organisation and every postholder we list carries a unique Oscar number. On deletion the record is archived and the number is never reused.
Oscar org types
- Level 1 — broad area of public services (e.g. Local Government)
- Level 2 — more specific organisation type (e.g. Local Authority)
- Level 3 — further detail where needed (e.g. English County Council)
Oscar functional categories
700 standardised functional categories are linked to posts — commonly more than one per post — and grouped into 36 functional areas for navigation and selectability.
Oscar seniority categories
- Chairman — most senior non-executive board member
- Chief Officer — the most senior executive/employed post
- Deputy/Assistant Chief Officer
- Clerk/Secretary — most senior administrative contact
- Top level director — reports directly to the Chief Officer
- Second level / Head of Service
- Third tier / Manager
Accuracy and analysis
A continuous six-month cycle keeps the average postholder record age at just three months.
Live research activity
See the maintenance work happening in the database
These figures are pulled from Oscar's live database update feed and refresh automatically.
—
Postholder updates in the last 7 days
—
Postholder updates in the last month
Why this matters: these figures show the volume of ongoing research and change being applied between scheduled full research cycles.
97%+
Measured core-data accuracy
3 mths
Average record age
500,000
Annual updates
8
Date fields tracked
Users should expect an accuracy rate above 97% across core post and entity data points. This is tested monthly through our own interactions with the database and our monthly e-newsletters.
Record age is one of our key indicators. Date added, date last changed, date last confirmed and date deleted are all maintained separately for organisations and posts.
An internal live report analysing the age profile of each dataset is reviewed weekly by the research department heads, with outliers passed to the relevant team for investigation.
How we group changes
- New — a post or organisational record has been added
- Changed — a record has been changed in some way
- Deleted — a record has been removed from circulation and archived
Last 90 days
Postholder change analysis
A rolling analysis of changes made to postholder records over the last 90 days.
| Database area | Deleted | Changed | New | Total |
|---|---|---|---|---|
| Schools | 1,770 | 14,531 | 2,584 | 18,885 |
| Parish Councils | 10 | 76 | 22 | 108 |
| NHS | 3,635 | 24,567 | 4,378 | 32,580 |
| Further and Higher Education | 214 | 2,954 | 280 | 3,448 |
| Local Government | 354 | 24,895 | 520 | 25,769 |
| Central Government | 567 | 3,702 | 680 | 4,949 |
| Care Homes | 1,505 | 8,356 | 1,111 | 10,972 |
| Housing Assocs | 284 | 1,829 | 298 | 2,411 |
| MP's and Lords | 19 | 1,486 | 50 | 1,555 |
| Dentists | 911 | 9,929 | 1,499 | 12,339 |
| Total Changes | 9,269 | 92,325 | 11,422 | 113,016 |
Live, compliant and transparent
Whatever you view or consume is the latest version of our data, with permissions reflected in real time.
GDPR
EU/UK compliant
PECR
Compliant
Daily
TPS screening
2bn
Downloads and updates
Live data means that whether you browse online or consume data through feeds and downloads, you always see the latest version — and the latest permissions. Download and feed users should take updates at least every 30 days, and ideally more frequently.
Coverage of personal data
- The name of the holder of an official post or position
- Any elements of corporate subscriber email addresses that identify an individual
We only list these data points where they have been given to us with our usage and purpose disclosed, or where — subject to additional checks — they are published by the employing entity.
Every data subject has real-time access to their own control panel and preference centre, giving them full control over which personal data points are recorded, where they are available, and which subscribers can access them.
Additional compliance
- A Data Transparency document shared with data subjects at, or within 30 days of, research
- Legal status recorded for every organisation — corporate vs non-corporate
- For non-corporate bodies, all named and direct emails are removed from access
- Subscriber-level email opt-out and suppression list management
- A full TPS/CTPS screen run daily across the entire database
Technical & supporting information
Go deeper into how Oscar data is structured, categorised, delivered and governed.
You matter
The community around the data makes it better for everyone.
We see all users of our data and systems as a community. Although the vast majority of our maintenance is proactive, there is always a valuable reactive element.
We value and encourage feedback, notifications of inaccuracies and details of forthcoming changes, and always pass these to the appropriate researcher for review, validation and update. Because our platforms are live, those changes are often seen and consumed within hours.
The net result of feedback from countless data subjects, public bodies and commercial users is a better dataset for all.
See how the methodology translates into usable public sector data
Explore the database coverage, build a selection or speak to the Oscar team about the audience, structure or delivery route you need.
