Housing Systems Depend on Infrastructure. Including for Housing Data.
Housing Systems Depend on Infrastructure. Including for Housing Data.
A key path to a better housing system runs through housing data and information
On my now annual list of CA Housing Priorities, I’ve included a specific shout-out to one of the most niche of housing issues: the data infrastructure that underpins our housing system. As I wrote in both 2022 and earlier this year:
We need to think more about the basic data and information systems that help us find housing, learn about our housing rights, or help us develop and make decisions about housing policy.
This issue is slowly creeping into the minds of housers beyond just those of us who work on data infrastructure. In 2022, California’s Housing and Community Development Department (HCD) and Office of Planning and Research released the first ever state Housing Data Strategy as part of the State Housing Plan. This housing data strategy was mandated by AB1483 (Grayson, 2019), an important sign that both sides of our government with responsibility for housing - the legislature and the administration - take housing data seriously. The past few years have seen the development of many new tools - tools developed by major global tech companies, small start-ups, foundations, large nonprofits, activist groups and regional and state governments.
So What is Housing Data Infrastructure? Do You Mean Housing Data?
Data infrastructure is the websites and software and data portals that give us access to critical housing information. What matters here isn’t (just) the data, but how we, as users and practitioners, access that data. It doesn’t matter if the data exists if you can’t find or analyze it, or if it is in a form you can’t understand.1
I like to think about housing data infrastructure in two big categories:
Tools that help us make decisions about policies and programs
Tools that help us find housing opportunities and housing information
Each of these areas has promising developments and some very specific challenges.
Category 1: Tools for Decision-Making
When a policymaker proposes a change in housing or land use law, how do we anticipate the impacts? The answer at the California state level is that we have very little idea. We don’t do systematic spatial analysis of state housing and land use legislation’s impacts. If you have seen any analysis of a bill - for instance, last year’s AB2011, or 2021’s SB9 - chances are private organizations or individuals, or academic institutions who happen to have funds available for that specific analysis conducted it.
The overwhelming majority of housing and land use bills at the state and local level receive no real spatial analysis. Our overworked Legislative committee consultants will do legal and political analysis and bills with a fiscal tag will get some sort of financial analysis. But what about the impact on actual housing production, preservation or protections? What will this do on the ground in our communities? What impact will this have on our homebuilding industries and construction workforce? In most cases, we have no idea beyond back-of-the-envelope guestimations. We’re basically flying blind.
Part of our analytical challenge is technical, part is financial, and part is political. On the technical side, we don’t even have the basic data infrastructure to do these analyses in the first place. Take zoning changes. Many bills these days attempt to change zoning codes. If you want to understand the impact of changes to these very complex codes, you need these codes digitized in ways that fit with spatial analysis software. If you are the state legislature and you are proposing a change to zoning across the state, you need ALL the zoning codes from California’s almost 500 jurisdictions digitized. Guess what? We don’t have this. That is why a state zoning database is a top priority for the Housing Data Strategy.
Even if our databases were perfect, these are super sophisticated analyses requiring highly trained teams and skills which often go beyond the basic set that many professionals possess - myself included. We can overcome this with specialized software, but this software needs to exist and you have to have access to and the training to use it. Companies like MapCraft and UrbanFootprint have the capacity and the tools, but we don’t have a structure whereby their tools can be used in any systematic way by public officials, advocates or anyone else. The Terner Center / Terner Labs team is diligently working on a dashboard to make this easier, but we still have a long way to go to set up an analytical system that really reaches all decision-makers - from legislators to activists - and enables them to use the most sophisticated tools we have to understand our decision’s impact on actual housing. Even better, imagine if we could analyze a few different scenarios for legislation based on how we write the rules?
A truly democratic analysis of policy changes requires the tools and data and resources to do the analysis, and a widely distributed skill set so that every community has a data scientist who can ensure that a more ‘data-driven’ approach is more democratic rather than elitist. All these things are possible, but unfortunately most of our analytical resources aren’t spent on legislation. We spend them instead on endless environmental impact reports for individual projects, rather than on systematic analysis. We waste millions of public and private dollars every month on project analysis to satisfy legal challenges, rather than using that money to design a better housing system for California.2
We also tend to keep adding to the analytical mandates that local and regional governments must meet to satisfy state law. Housing elements, affirmatively furthering fair housing requirements - we keep mandating new analysis without really building a smarter, more efficient and more effective analytical system. It thus remains fragmented, and mostly looks in the rearview mirror. Cities are required to do their own analysis about the past and provided with little incentive or assistance to analyze what happens in the future if we take certain actions.
Finally, and perhaps most critically, housing data infrastructure is essential to any hopes of implementation and enforcement of housing laws. We will never have proper enforcement of AB1482’s landmark rent regulations unless there is a rental registry or similar system which allows public officials and advocates to see what is happening. We know this in part because we have built a data tool to help officials and advocates track another key state law - the Housing Accountability Act and related rules around Housing Elements and Regional Housing Needs Assessments. HCD’s Annual Progress Report dashboard has helped advocates seeking to track their city or multiple cities and their progress towards meeting state housing mandates. We now need to expand this approach so it covers the full range of protection, preservation and production goals.
Category 2: Tools for Citizens
The other set of tools are for citizens looking for housing, housing assistance, information about their housing rights, or somehow trying to access the actual housing system (as opposed to the system where we make the rules). Included alongside citizens looking for housing are all the social workers, housing counselors, real estate agents, friends, family members and neighbors who help the less technologically savvy of us find this information.
Take for instance tenant rights. Not only do we struggle to enforce AB1482, we struggle to get information about tenants rights to the people that need them. State laws interact with local rent laws, and the types of protections, rights and resources available to tenants varies dramatically by jurisdiction and building type. As a citizen, how am I meant to learn about my rights? As a housing counselor or tenant organizer, how do I help someone learn theirs?
Spend a bit of time googling around and you will realize that we didn’t set up a data infrastructure system alongside AB1482. Activists and foundations were forced to scramble to create tenantprotections.org as a pathway to find locally specific information. This comes at the same time as the infrastructure designed to help support tenants and homeowners with rent and mortgage payments struggled during the pandemic.
All throughout our housing system, we don’t have adequate information infrastructure to help people get housing or keep housing. Want to find affordable housing? In most of California, that means finding your way to a million different waiting lists. San Francisco tried to change this, working with local tech firm Exygy to create DAHLIA, a single portal for affordable rentals and homeownership opportunities. DAHLIA is now the basis for MTC/BAHFA’s Doorway project, which will hopefully bring this opportunity to all 9 Bay Area counties. The rest of California will have to wait - AB312 (2023, Reyes), which would have created a statewide system, died in appropriations.
The challenges go beyond public sector tools. For those of you following my work on Multifamily Homeownership, one challenge is that the major real estate sites don’t allow you to search for a Tenancy-in-Common or most housing options beyond a condo or a single-family home (you can find them, but generally by accident - shout out to Compass for at least having an ‘other’ category). Even when our housing system provides units and rights and support, we simply don’t have the data infrastructure or information system to ensure that the people who need to access them can do so.
Improvements and Changes
So what do we do? The Housing Data Strategy, which I encourage anyone in this space to read, has important steps for building better data streams from local jurisdictions, encouraging cooperation and building out technical assistance programs. It’s a vast and connected field, and there are dozens of different data infrastructure projects being built simultaneously, including many referenced above.
In the meantime, there are some important things we can all keep in mind - especially those of us who work in this subsector of housing and data science are connected in some way.
We need more ambitious cross-sector collaborations, especially between public, private and nonprofit sectors, building on what MTC/BAHFA + Exygy + Google.org (and other partners) are doing. There is no way California can build a housing data infrastructure system capable of transforming housing without our tech sector as a key participant. But neither can they do it alone. We need a specific strategy for ensuring that small tech companies, activist non-profits, foundations and public sector agencies are able to work with big tech companies - each have a role to play.
We need next level user research. Do we really understand who the audience for our tools is, and are making sure we’re crafting what they need? Sometimes we build tools that are too complicated for citizens and too simplistic for professionals. Other times we imagine that citizens will engage with tools directly, when we should target intermediaries like social workers and tenant organizers.
We need more coordination and publicity. Too many tools just sit in the ether, and don’t get used because nobody knows they are there or don’t know how to use them.
We need to figure out more sustainable tool building. We often build tools and then they quickly become outdated, as we don’t have plans for long term maintenance and growth of tools.
We need to think even more about training programs, and the ways every community can have access to data scientists and members who become data scientists or people who can use tools - whether for decision making or for access to housing. Much like every community needs a good lawyer and accountant, we now all need data scientists.
We need to think more about how universities can play a central role in hosting and coordinating this infrastructure. Universities are the natural place where tech, government, nonprofit and philanthropy can come together to build data infrastructure, and to train the next generation of builders and users more holistically. We need housers who know data and data folks who know housing. Otherwise neither will help each other.
We need to serve all of California. The infrastructure gaps in rural and small metro California are even more severe than in the big metros, and we need a system - likely involving collaboration between Councils of Government (COGs) and CSUs/Community Colleges - to ensure rural California has equitable access.
One nice thing to report is the growing energy for cross-sector collaboration in this space. Recently, my partners at SJSU’s Institute for Metropolitan Studies (IMS) and the Spatial Analytics and Visualization Institute collaborated with the TechEquity Collaborative to bring together a cross-sector workshop on this issue. We had activists, small tech entrepreneurs, regional government officials, state government reps, funders, policy wonks, lawyers and data scientists sitting together and finding common ground on these issues. We’re a long way from building a true 21st century housing data infrastructure system, but I am happy to report we have more and more good folks from across the data and housing spectrum engaged and working to make change.
A critical first step is simply appreciating what this issue is to housing and why it matters. A few weeks ago, an esteemed colleague who has worked in housing for decades found out about my interest in housing data infrastructure. He was a bit puzzled - why would I care about something like this, when there was so much else to do in housing?
I hope it’s clear now. My colleague can’t do their job without housing data infrastructure, and their job isn’t as effective on the ground without it. Data infrastructure may not be as important to housing as sewer and water, but it’s not far off in our current world.
A more accurate term would probably be Housing Information Infrastructure, as information is what we’re really after. But data sounds better, and people are into data these days, so data it shall be (for the moment!).
We also burn through endless hours of really talented and dedicated research analysts and planners whose work gets buried in EIRs. Imagine all that talent and all that time channelled into problem solving and scenario testing for our communities, regions and state? We might actually be able to make change.