It's been a long time since my last post. I have a personal project I've slowly been chipping away at, but there has simply been so much work that it's hard to get motivated for side-projects.

Last year, I was assigned to a project called the Property Inventory Management System (PIMS). It was a stack of React, ASP.NET, and MS SQL Server. I really didn't know anything about .NET or SQL Server, so it was a bit of a learning curve. Right away, it was pretty clear that there were a lot of issues plaguing the application. The number of endpoints that were in use but unfinished by the original team (contractors, now long gone), the horrifying frontend states that seemed to be altered from anywhere. It was a lot to take in, but there was already talk of a rebuild, so by the time we got to December, we were starting from the ground up.

I don't want to dwell on the specifics of what it took to put together this incredible rebuild, but I can't bring myself to continue without paying acknowledgements to the amazing team that worked on it. Our development team was only five people at the most, and everyone has left their mark on the project.

We built an entirely new API using Express, a redesigned frontend from the ground up, and moved everything away from MS SQL Server and into Postgres/Crunchy. It was a lot of work, but the results are so clearly an improvement that it would be hard to argue that it wasn't worth the time. No longer does it look like it was made in the early 2000s instead of 2020 when it was built. Developers can actually tell what's going on now in the backend without being surprised by To-Do notes from four years ago. I can't share the actual site, because you would need permissions to access it anyway, but I'll share this screenshot of the map, which I'll talk about more later.

I would say that it's worth checking it out on GitHub if you're into the tech-side of things. You'll find a link at the end of this post. There's a lot of documentation there that explains it in better detail than I would here. Instead, I would rather talk about what I learned from this giant undertaking.

There were a lot of occasions where we needed a timeline for leadership, other groups, or even just our own planning. I'm apparently an optimistic estimator because, early on, we tended to overshoot my estimates easily. I suspect one reason for this was that I was only thinking about the best-case scenario. Surely this task will only take a day, but then we didn't see progress on it for a week. At this point, I think I'm a lot more cautious, but my manager also made me realise something when he pushed back against these deadline questions. We run an Agile shop. It's an iterative process, and we're done when it's ready. This works because we already had a working application in place, and it's especially easy to think this way when there's no reason to rush the project.

We had a reference in the old system, but there were so many pieces that seemed to have been made with features in mind that never came to fruition. Because of this, we leaned hard on the product owners to understand the features that the application really needed to have. For the most part this was good, but there were a few cases where a feature would be done for several weeks and even in use when we would hear that it also needs to handle some additional cases. Usually, this would have been easier to tackle if we had known about it from the beginning. I'm going to make sure these requirements are clear cut in the future to hopefully avoid this.

PIMS uses a lot of external data sources. There are multiple map layers, data from properties coming from sources like BC Assessment and the Land Title and Survey Authority, and brokered login options such as the BC Services Card. In theory, this is great. We can use their services and it's all risk that's outsourced to these groups instead. In practice, I found that it added a lot of risk in terms of timelines and reliability. Weeks turn to months as we wait for responses from outside groups. Often we are not connected with the personnel to answer our technical questions. At times, it makes me want to shirk any outside connections and just handle these cases on our own. While I know this is pretty much impossible for a lot of these data sources, it can be tempting. How to work with the people to get these services running is something I feel like I have practice with now. Getting through the bureaucracy that comes with that is still an ongoing battle.

Moving on to more tech-focused topics, Leaflet was probably my favourite experience of the project. I love geography and maps, so this was a handy tool that allowed for easy spatial representation of our data. It's not the most feature-complete option for working with spatial data, but it is clean and quick. I managed to build a number of features on top of our property map, including different kinds of popups, filters, and clustering behaviours. I'm really looking forward to trying this out in a personal project one day with a custom tile set.

We knew that we wanted an ORM to help manage our database access, but I hadn't really worked with any of them before this. It definitely made a lot of database tasks easier. If you're not doing anything complicated, it makes creating your entities and migrations very painless, and performing CRUD operations is exactly what you want it to be most of the time. Sometimes, however, I found our needs for the project resulted in us abandoning the repository manager system in favour of their query builder, which feels a lot more like just writing SQL again. There were some cases where TypeORM would do something strange in the background, making queries much less optimal than they should have been. It's not the perfect tool, but I'd probably use it again. PostgreSQL felt like a big step up from SQL Server. Being able to avoid .NET and Entity Framework made working with migrations and database changes so much easier.

Express is not new to me. It is definitely my most comfortable API framework, but recently I found myself working to redo the entire authorization system we had set up. It wasn't by choice, but was instead a requirement of including a new form of authentication. Tackling this task really opened up my understanding of how middleware can be used in Express to blow open the doors. My current example was with how we checked for user roles. When a user sent a request, we would have to get the user's username, make the call to the database for their role, then see if it matched a required role. This might be done in a controller, but there was no guarantee that something similar wouldn't be needed in a later piece of code. This could result in multiple database trips all for the same info. Writing a new piece of middleware allowed this all to be done once. We could get the user, check if they have the roles needed to access this particular route, and then tack on that user's information to the request for other checks later on. No repeating the same requests. I'm really proud of this change. It seems like the obvious thing, but it allowed us to cut down on so many queries.

PIMS is still evolving, but we're at the point where things are slowing down. Team members are slowly being dispatched to other projects, and it seems like I'll have new work as well in the coming months. Still, PIMS was my first team project. It really took me from the mindset of solo developer to team member. I have a few little surprise features I'd still like to get in before the page turns on this chapter, and we're currently undertaking authentication using the BC Services Card.

Again, I'd love it if you'd check it out. It's not the most complex application, but it represents a significant chunk of my time with government so far.