Summarizer

> I wonder if, 10 years from now, LLMs will still be answering questions that were answered in the halcyon 2014-2020 days of SO better than anything that came after? I've wondered this too and I wonder if the existing corpus plus new GitHub/doc site scrapes will be enough to keep things current.

There's another significant forum: GitHub, the rise of which coincided with the start of SO's decline. I bet most niche questions went over to GH repos' issue/discussion forums, and SO was left with more general questions that bored contributors.

> SO was by far the leading source of high quality answers to technical questions We will arrive on most answers by talking to an LLM. Many of us have an idea about we want. We relied on SO for some details/quirks/gotchas. Example of a common SO question: how to do x in a library or language or platform? Maybe post on the Github for that lib. Or forums.. there are quirky systems like Salesforce or Workday which have robust forums. Where the forums are still much more effective than LLMs.

Q&A isn't going away. There's still GitHub Discussions.

On the other hand, another week another JavaScript framework, amirite? There continues to be new stuff to ask questions about, but stack overflow failed to be the default location for new stuff. I guess now there's more discussion directly on GitHub and discord.

There is also github issues discussions now which also helped in asking these niche questions directly to the team responsible. I dont ask questions about a library on SO I just ask it on the github of the library and I get immediate answers

Absolutely 100% this. I've used them on and off throughout the years. The community became toxic, so I took my question to other platforms like Reddit (they became toxic as well) and elsewhere. Mind you, while I'm a relative nobody in terms of open source, I've written everything from emulators and game engines in C++ to enterprise apps in PHP, Java, Ruby, etc. The consistent issues I've encountered are holes in documentation, specifically related to undocumented behavior, and in the few cases I've asked about this on SO, I received either no response and downvotes, or negative responses dismissing my questions and downvotes. Early on I thought it was me. What I found out was that it wasn't. Due to the toxic responses, I wasn't about to contribute back, so I just stopped contributing, and only clicked on an SO result if it popped up on Google, and hit the back button if folks were super negative and didn't answer the question. Later on, most of my answers actually have come from Github,and 95% of the time, my issues were legitimate ones that would've been mentioned if a decent number of folks used the framework, library, or language in question. I think the tl;dr of this is this: If you can't provide a positive contribution on ANY social media platform like Stack Overflow, Reddit, Github, etc. Don't speak. Don't vote. Ignore the question. If you happen to know, help out! Contribute! Write documentation! I've done so on more than one occasion (I even built a website around it and made money in the process due to ignorance elsewhere, until I shut it down due to nearly dying), and in every instance I did so, folks were thankful, and it made me thankful that I was able to help them. (the money wasn't a factor in the website I built, I just wanted to help folks that got stuck in the documentation hole I mentioned) EDIT: because I know a bunch of you folks read Ars Technica and certain other sites. I'll help you out: If you find yourself saying that you are being "pedantic", you are the problem, not the solution. Nitpicking doesn't solve problems, it just dilutes the problem and makes it bigger. If you can't help, think 3 times and also again don't say anything if your advice isn't helpful.

I also wonder if GitHub Discussions was also a (minor) contributing factor to the decline. I recall myself using GitHub Discussions more and more when it came to repo specific issues. The timeline also matches: https://github.blog/changelog/2020-12-08-github-discussions-... https://github.blog/news-insights/product-news/github-discus...

Do we have any stats for the number of GitHub discussions created each month to compare to this?

This type of thing often lives in the issues / discussion tab of a github repo now a days, for better and worse.

Yuck. I don't know if it's just me, but something feels completely off about the GH issue tracker. I don't know if it's the spacing, the formatting, or what, but each time it feels like it's actively trying to shoo me away. It's whatever the visual language equivalent of "low signal" is.

I think most relevant data that provides best answers lives in GitHub. Sometimes in code, sometimes in issues or discussions. Many libs have their docs there as well. But the information is scattered and not easy to find, and often you need multiple sources to come up with a solution to some problem.

While AI might have amplified the end, the drop-off preceded significant AI usage for coding. So some possible reasons: - Success: all the basic questions were answered, and the complex questions are hard to ask. - Ownership: In its heyday, projects used SoF for their support channel because it meant they don't have to answer twice. Now projects prefer to isolate dependencies to github and not lose control over messaging to over-eager users. - Incentives: Good SoF karma was a distinguishing feature in employment searches. Now it wouldn't make a difference, and is viewed as being too easy to scam - Demand: Fewer new projects. We're past the days of Javascript and devops churn. - Community: tight job markets make people less community-oriented Some non-reasons: - Competition (aside from AI at the end): SoF pretty much killed the competition in that niche (kind of like craigslist).

The obvious culprit here are the LLMs, but I do wonder whether Github's social features, despite its flaws, have given developers fewer reasons to ask questions on SO? Speaking from experience, every time I hit a wall with my projects, I would instinctively visit the project's repo first, and check on the issues / discussions page. More often than not, I was able to find someone with an adjacent problem and get close enough to a solution just by looking at the resolution. If it all failed, I would fall back to asking questions on the discussion forum first before even considering to visit SO.

SO peaked long, long before LLMs came along. My personal experience is that GitHub issues took over. You can clearly see the introduction of ChatGPT in late 2022. That was the final nail in the coffin. I am still really glad that Stack Overflow saved us from experts-exchange.com - or “the hyphen site” as it is sometimes referred to.

GitHub Issues and Disscussions + searching the code base, fetching the docs and some reasoning on top. Maybe even firing up a sandbox VM and testing some solutions.

Now that StackOverflow has been killed (in part) by LLMs, how will we train future models? Will public GitHub repos be enough? Precise troubleshooting data is getting rare, GitHub issues are the last place where it lives nowadays.

This entire thread is fantastic. I felt nostalgic, angry and then concerned all at once. I love LLMs. But I miss SO. I miss being able to have that community. How do we bring it back? If anyone from the Stack Overflow team is reading this (I assume you are): what’s the plan? My take: stop optimizing for raw question volume and start optimizing for producing and maintaining “known good” public knowledge. The thing SO still has that Discord and LLMs don’t is durable, linkable, reviewable answers with accountable humans behind them. But the workflow needs to match how devs work now. A concrete idea: make “asking” a guided flow that’s more like opening a good GitHub issue. Let me paste my error output, environment, minimal repro, what I tried, and what I think is happening. Then use tooling (including an LLM if you want) to pre check duplicates, suggest missing details, and auto format. Crucially: don’t punish me for being imperfect. Route borderline questions into a sandbox or draft mode where they can be improved instead of just slammed shut. Second idea: invest hard in keeping answers current. A ton of SO is correct but stale. Add obvious “this is old” signaling and make it rewarding to post updates, not just brand new answers. Last thing that I don’t see an easy answer to: LLMs are feasting on old SO content today. But LLMs still need fresh, high quality, real world edge cases tomorrow. They need the complexity and problem solving that humans provide. A lot of the answers I get are recycled. No net new thinking. If fewer people ask publicly, where does that new ground truth come from? What’s the mechanism that keeps the commons replenished? So… TLDR…my question to this group of incredibly intelligent people: how does SO save itself?