When I worked for a huge leading IT company, I was often tasked with writing “quick n dirty” scripts to collect data reports. These problem areas would be fixed with either new scripts or GPOs. Along side this, some of these reports lead into compliance reports for external auditing.
Now I have a bit of time on my hands, I’d like to make a master dev ops tool. The problem with what I did before, we would end up with several reports in CSV format floating about / edited / not deleted on various file stores / emails etc.
I’ve heard a little about DSC and wonder what are your thoughts about when you should / shouldn’t use DSC?
I want to develop a tool which scrapes / presents the data into a database / with a front web interface. I’ll have the web tool, scrape AD for server objects and add it into a database. From here, I want to contact these servers and then collect further information eg, what Drives / space there are. What local groups and accounts exist. What password policies are in place. What event logging settings are etc. What GPOs are applied etc and what AV is in place. As well as many other useful bits of data. I’d like to store these findings in the database, so I can run queries against to produce web based results — this then can be used by the rest of the team … eg, which servers don’t have AV installed, which servers aren’t password compliant etc.
So before I start converting my scripts into advanced functions or DSC … what would you consider?
I hope you’ll ask a question, too!
So, I mean, you’re going to rebuild System Center Configuration Manager. Basically, at least in part. And the difficulty with that is creating sufficient scale. SCCM has a ridiculously complex infrastructure because of scale. You can’t, for example, just have 5,000 servers dumping data directly into a SQL Server database, because the database won’t scale as you grow (or, at least, it will be prohibitively expensive to make it scale). That’s why SCCM has its complex system of collectors, readers, and so on, and why nothing in SCCM happens instantly. It’s all designed for scale. And yet, with all that, SCCM still can’t scale to cloud-scale, which would entail millions of systems, not a paltry 100,000 or so.
I think you’re still thinking in 1990s terms. I don’t want that to come across as mean; it’s a very common mindset. Sure, you’re trying to automate a lot, but you’re automating what we did in 1995.
Here’s a key phrase:
I’d like to store these findings in the database, so I can run queries against to produce web based results — this then can be used by the rest of the team … eg, which servers don’t have AV installed, which servers aren’t password compliant etc.
Thing is, in a proper DevOps environment, I don’t care about these things. I have to assume that any of my systems would be blown away and rebuilt from scratch. That’s probably happening every few minutes, which is what containers enables. Point is, you’re trying to track the information that makes each machine a Unique And Special Snowflake, when in a DevOps world, I’m looking to eliminate snowflakes and make everything into Cows I Don’t Care About Personally.
The point of DSC, Puppet, Chef, and the like (and even GPO in a rudimentary way) is to locally ensure that my desired configuration is active. I need to trust those tools, or there’s no point having them. I don’t run around and collect evidence of compliance, because that’s wasting effort. I collect whether or not the system is compliant, which DSC, Puppet, Chef and the like can all already report on, and I call it a day. If a machine stops being compliant, I’m inclined to blow it away and start over with a new machine that’s more obedient. Why troubleshoot the problem when starting over is so much faster and more satisfying?
External auditing has to become, “here is the desired configuration, and here is the indicator that the system is in compliance with the desired configuration. You’re done auditing.”
I realize that the trust-in-technology required by all of the above is hard to achieve, and that’s why many organizations can’t ever get there. It’s a big, big leap. But that’s what DevOps is; you can’t simply automate the 1990s-era “manage each server individually” approach and call it DevOps. That’s like putting high-test gasoline into a Hyundai and calling it a Ferrari. It ain’t true. You’ve still got a Hyundai.
Now, suppose we take “DevOps” out of the equation. Suppose you’d said:
Now I have a bit of time on my hands, I’d like to make a master server information repository.
Sure, okay. If that’s what you want to do with your time, then your approach seems valid. I mean, it’s the same approach multiple commercial tools already take, and you will find yourself with some scaling problems at some point, but that’s fine. Dumping data into SQL Server and generating reports in SQL Server Reporting Services is totally cool.
I’m just not sure it’s what I’d spend my free time on. As I harp on in Be the Master, it’s really crucial to Define Your Success and Define Yourself. You need to know who you are, and who you want to be when you grow up. Free time is a deeply precious commodity, and you should only spend it helping to move yourself toward your success. Is this project where you’d spend your time, if you had a clear picture of “who you want to be when you grow up?”
Not knowing anything about who or what that might be, and assuming that your success involves something around DevOps and automation, I might suggest instead spending time helping a project like Tug, maybe with a solid layer for DSC report collection and reporting tools. You’d be contributing to an open source project, and you’d get your name “out there” in a big way. You’d be helping to solve some of the problems you’ve outlined, like better reporting of system compliance, in a way that’s more 21st century. And if not Tug, something that helps move the industry forward, rather than allowing companies to just remain mired in their 1990s management techniques.
Hope that helps a bit ;). Do feel free to hop in the forums at PowerShell.org if you’d like to discuss/debate!!