Is copilot slowly getting worse?

[bdanmo]

Select Topic Area

Product Feedback

Body

When I first started using copilot, it was incredible. Over the months, it has slowly become less and less usable. This is happening in so many disparate ways, that it is tough to share specifics. To summarize, I used to comment and start a new-line (say in python or powershell), and I would get something very useful based on the comment I wrote. That may be a single line or a whole function or object/dictionary. It used to feel powerful, and now it feels... broken. I add a comment, and the next line suggested by copilot is that same comment repeated again, or another comment with a different instruction based on what copilot thinks I'm trying to do, rather than a line code doing what I said in the comment. This may repeat until I skip a line. Then it will attempt to write code line by line, but that will quickly turn into nonsense or it will stop making suggestions. I know copilot isn't supposed to do all the work, and I'm not trying to make it do that, but I've found that how it functions and how I can use it has changed drastically. The suggestions I get now, whether working off of a newline after a comment or in the context of the rest of the code, are at best less sensible and less useful than they used to be. At worst, and more frequently, it's completely unusable. Anybody else experiencing this?

[milindaShehan]

Yes, some developers are experiencing GitHub Copilot becoming less useful over time. This may be due to Copilot still being under development, the fact that it is trained on a massive dataset of public code, or its difficulty understanding the context of your code.

[techarchitect1]

Most of the time I get frustrated with Copilot. During the preview time it was amazing now it is less helpful for me.

[langaads]

Sharing the same experience for quite a while and it became even worse since then. It's soooo slow and generates barely 1 line of code that is incorrect 50% of the times even when the context is right there in the same file. It's not worth it's price anymore.

[Claquettes]

Maybe it has to do with this:
"Copilot loses an average of $20 a month per user, according to a Wall Street Journal report, with some customers costing GitHub as much as $80 a month."
src: https://techcrunch.com/2024/04/29/copilot-workspace-is-githubs-take-on-ai-powered-software-engineering/

[justlevine]

Good to know it's not in my mind. Autocomplete usually starts to fritz around 30 minutes into a coding session, and Copilot Chat can't even retain the context after 2 or 3 replies. That includes code highlighted or referenced by @\workspace.

[matrs]

copilot is so much worse the last month or so, it's almost unusable for chats. It continues repeating the same bs after telling it to not repeat the same bs again and again. Months earlier, i could talk to it , give it some information/cntext and a bunch of times its answers will be correct or at least helpful, now now it's very stupid to be honest.

[a-lebailly]

copilot is so much worse the last month or so, it's almost unusable for chats. It continues repeating the same bs after telling it to not repeat the same bs again and again. Months earlier, i could talk to it , give it some information/cntext and a bunch of times its answers will be correct or at least helpful, now now it's very stupid to be honest.

@matrs Thkkkkkk !!! I've been feeling the same way for a few weeks now. The chat has literally become a moron, and I'm weighing my words carefully ! It's strange because a few weeks ago, I thought it was as good as ChatGPT, but now, with 3 context files, it's lost its way and doesn't make any sense, even on the most basic things.

[imDarshanGK]

yes

[FeMaffezzolli]

It's not helpful anymore for me. Any suggestions for replacement? Codeium maybe?

[ShayBox]

My guess is that they have had to remove public datasets as requests and legal action is taken, remove projects using certain licenses, etc.

[henryobiaraije]

This is getting worse every day.

[OAkyildiz]

I would say it's getting rapidly worse. At this price point with this performance, it's a scam

[dimitardimitrovyotpo]

Just renewed for a year and it has already started hallucinating even for Java.. suggesting wrong dependencies, losing context, worse suggestions using wrong types in a strongly typed language..
On an yearly basis I cannot see it giving me back 100$ worth of time in its current state, so definitely a bad deal.

[ShayBox]

Copilot was amazing during the initial trial period, and even within a few months of being released, but as of now whatever LLM they switched to instead of their original specialized solution is terrible and not worth it. Ollama + Continue.dev + deepseek-coder-v2:16b, codellama:34b, and phind-codellama:34b work very well for me for free.

[gentstats]

I've been using Copilot intensively and I've been very happy with the result, it's been a great help. Now, without realizing it, I've found myself wasting time correcting suggestions, getting frustrated because it loses context or starts to hallucinate...
To top it off, I'm writing a markdown and it starts suggesting me plain html.
So I've come here to complain, because if they don't offer us a solution, this will be the last billing cycle.

[RomainLAU]

I just lost like 30min trying to explain to it that React doesn't work if a hook is called inside a function in a component, and it repeated again and again "i'm sorry, here is the code fixed: [same error]", it's just annoying to use now, I'm surprised when it finally succeeds to give me an helpful solution to my issue, I would be glad if someone had any other tool to suggest, if possible free, at least I know I'm not the only one in this situation !

[RomainLAU]

I tried every way to make it understand its error, I asked him some question to see if he understood, he seemed to, I asked him to fix his error, and repeated again the same thing, as if everything I just said never happened...

[RomainLAU]

Also, at the beginning, like last year, I was using a lot ChatGPT for my code, then I saw Copilot was way better for the code, so I stopped using ChatGPT, but now ChatGPT is like 1000x more useful than Copilot, even with the free version !

[LouisVA]

In the last month it helped me a lot through some rough libraries that I have no experience with. But this week Copilot just became a joke. It can't use cv2.rectangle (from the python cv2 library) correctly. Like wth guys ?

[Gpy1123]

I am having the same experience, at first it was really useful but now both the context of the code and its actual quality have gotten... plain bad.

Sometime co-pilot generate bugs trying to access class property that just don't exists. It is getting in the way of IntelliSense by completing your code with non-existing references. When your head is in the code it is easy to press enter or tab and just get plain wrong code generated.

At other times it just generates broken code structure, like missing closing } or ; or not finishing a try catch block properly.
Did it learn too much from human code to the point where it starts reproducing human typos and mistakes or what?

It feels like reviewing my interns code at this point.

[prdai]

I have actually experienced a similar situation. It mainly seems like the prompts are not appropriately followed, I suppose.

[BradenGilbert]

I made a GitHub account just to comment on this...

I use it for work in the environmental industry, I enjoy it being able to take tables and complex lab reports and then pull in the specific wording of laws and regulations so that there is no misinterpretation. Over the past year plus it has slowly become extremely shallow and lazy.

I used to be able to ask it "List all states environmental agencies and link their websites" or "Provide a list of all the city in Texas that utilize Bonfire for procurement". I would get a result that would have taken me an hour or two to create myself. Now it will not link their websites unless it can hyperlink it as a reference at the bottom.

The procurement website listing is a great example, it will only list the same cities and its actually just faster to google them instead of even ask...

[BradenGilbert]

I assume they turned down the depth at which it searches to a level that is more affordable, but less useful. If true, they most likely targeted the adjustment in a way that only technical individuals would notice... That way they can still offer LLMs to the masses, while offering paid more useful services to those technical individuals...

How long until they market these "ai" solutions with metrics that communicate how much effort a specific ai model will utilize to get you an answer... I know it is done on the back end, but I am talking about general public level of understanding. like horsepower

[Jimgitsit]

Completions have really taken a nose dive lately. It has started suggesting non-sensical code and seems less context aware too. Used to be great and a huge timesaver. Now it's mostly noise. I rarely use chat, just completions. I have a pro account and all models enabled in site settings but only 4o Copilot is available for completions in Jetbrains and VS Code. I only started noticing a difference in the last 3 or 4 weeks though. Before that it was fine imho.

[zznathans]

I've reached the point where if I'm trying to do anything beyond the basic, I just disable the completions and use the chat feature only.

I try to create a simple list, it suggests an entire function. I ask it to do something complex, and it just hallucinates a method that doesn't exist.

I'm clueless about AI, but if I had to guess, it feels like they just started scraping and learning every single thing on the internet, whereas the earlier model was based on something more specifically curated? who knows

It's great for linting, making things pretty, and excessive comments. But I can do all that with open source tools and a CI job myself. So I guess its just a more complex google for me at this point.

[schoolsplay]

Yesterday I decided to cancel my subscription after two years. Code suggestions are getting worse and I got the feeling I spend more time correcting the suggestions then actually saving time. The free options seem to be at least on par with copilot.

[schoolsplay]

Could we experience the first signs of model collapse where the model is starting to poison itself?

[jan-bogaerts]

starting to get the same here. This is simply getting ridiculous: cursor at a list of function parameters, the function is used 5 lines below the cursor, with the exact same parameters as are declared at the position where the cursor is, the function isn't used anywhere else, yet the autocomplete comes super fast with a list of random variable names to add to the function, names that have no meaning what so ever to what the function does. frustrating.

[Demitrix2k]

Agreed, i end up wasting my time undoing stupid unnecessary context the AI infers on, or it suddenly looses a great deal of context, it really feels like time of day gives me different results, but combined now with these rate limits and the frustration with how bad the service is these days, i think im done.

[Jimgitsit]

I'm out too. Canceled my subscription. Trying out the intellij built-in AI. So far the completions are on par with what copilot used to be.

[nguyenlamlll]

After trying Cursor, I am out of words with Copilot. I have been living under a rock, a big ugly rock. Cursor ability is outstanding. One simple message, and it understands and creates everything I need. On Copilot, it barely understand my message... Not looking back.

[at-mc]

+1 The code agent has become significantly worse especially with the #codebase feature. I hope the team will consider the feedback from the community and fix it at least to the level of usefulness it had before.

[Sanjaya-Danushka]

I just want to say — I completely agree with this.
When I first started using Copilot, it felt like magic. I'd write a comment or just start a function, and Copilot would give me something really helpful — sometimes even a full function that was surprisingly accurate. But lately, it's been getting worse in exactly the same ways you've described.

Now when I write a comment, it often just repeats the comment back to me, or suggests a slightly different version of it, instead of actual code. Even when I give it really clear context, the suggestions are either off-topic, random, or just stop altogether. It doesn’t feel helpful anymore — more like a distraction.

I totally understand that Copilot isn’t supposed to do all the work, but it used to be a smart assistant. Now it just feels broken. I’m glad I’m not the only one noticing this, and I hope the team is able to bring back the level of usefulness it had before.

[scharada]

i have canceled my subscription. i decided to run a last test for this service.
at the time i am reading this. a code fix for a missing reference has been going for the last 4 hours with no action taken.
reading through the comments, i've seen and experienced most of the concerns. bad code suggestions, not following instructions, repeating same mistakes even after clearly writing a precise prompt, and what is bad, really bad, is the time it is taking to answer.
using it became a frustration more than anything else. paying for that does not seem right.

[at-mc]

I have a question/request for copilot code agent - the code agent has been refusing valid requests more and more lately. Is any feedback given to it taken into account? It seems like even for code within the same workspace (with explicit requests that require no external code changes and simply reorganization of our own code or changes to our own code), it blocks quite a lot with Sorry, the response matched public code so it was blocked and does not take valid requests. Can someone please address this issue? This adds on to copilot having becoming much worse than before.

[mwilson35]

I am shocked at how outrageously bad and useless it is. Cancelling immediately. I'm paying money for this? It's just going in circles for me, creating errors as it goes, unable to fix extremely simple ones. Anyone able to recommend anything else? I think I'll just cancel this and resub OpenAI.

[rfletchr]

mine declined in quality as soon as i purchased a yearly plan. since then its been getting slower and slower.

It feels like a bait and switch.

[Globolo001]

Can this actually be? I just purchased monthly and first day of the first month, Opus is like hangingly slow.
Maybe its because on new month the Anthropic servers are just capacity limited?

Will see wether it improves the next days (and also how Sonnet fairs)

[LouisVA]

I have no clue how they keep doing this. We have genius models that do genius things. And then a week later, your AI can't do the most basic thing. Refuses to actually listen to any form of command or actual instructions, just does whatever it wants ignores your inputs, screenshots. Unbelievable.

EDIT: Now another day later, even opus 4.5 has been effectively lobotomized. You spend 3 request credits and it writes 20 lines of code and calls it a day, totally ignoring part of the requested functionality. Before it would write me 5 files including extras I did not ask for. Unbelievable. I am happy my year subscription ends in January and I can go to Cursor. Does not matter what you use opus 4.5, gemini 3, raptor-mini. Either became stupid or super lazy.

[dannypike]

I'd like to add my agreement to this conversation. I've just spent over an hour with the AI telling me it's done something that it hasn't done at all. I looked into the messages that it was telling itself and it seems that it was deciding that I asked for it to do something else. It never used to do this.

Perhaps it is being overwhelmed by the volume of requests, or they've got a smaller model running or something? Telling it this worked for that particular problem:

Please store a permanent request to do what I ask, not what you think I might have meant to ask. I do not mind if we make a mistake because of my error, but I do not like it when you do not do exactly what I ask you to do.

If you do not understand something because it is ambiguous, please ask for clarification. Do NOT try to guess what I mean. Ask me if you are not at least 90% certain. Make this a permanent feature of your model when talking to me.

... but I don't hold out much hope that it will work forever. It does seem to have stopped committing and push changes to the repo, though. So that's something, at least!

I'm think I'm going to have to go back to chatting to local ollama models, for this year.

[ArrayIterator]

Maybe not getting worse! but WORST!

[dannypike]

Yes. I gave up on Copilot. And on Git, too. I'm using Forgejo now, and managing the backup of my own repo to offline storage. It's a real shame, but I'd rather spend time on getting that working reliably, than on dealing with major flaws in Copilot.

I find that Opus 4.7 with Claude Code works very well, as long as I use a clean language. So, I stay in Python/PySide6.

I have a lot of "rules" in my .config/CLAUDE.md file and Claude follows them very well. So, we have a very happy relationship; it generates very few bugs, and the bugs that it creates are often because there's some ambiguity in my instructions that I hadn't thought of. Or the source files have grown too large (I keep them to a max of 600-1000 lines to fit easily into the context window). That's quite acceptable to me.

The only "problem" that I have with Claude, is the (apparently) random way it eats up my weekly usage limit. I want to have a way of querying the number of tokens to be able to manage our usage, because I do not trust the values on the Usage tab of the account page. It will suddenly "jump" a lot for no apparent reason, even if it's Visual Studio just sitting idling, and I'm fast asleep!

But, despite that, it's still the best model I've encountered. Have you tried out the Ollama cloud, yet? I'm thinking about it in the background, but I'm nervous about using Chinese models. They, as a country, have an "interesting" reputation. What local model do you use? I only have an RTX 3090, so not much VRAM; I would love a NGX Spark (or eight), but I can't justify that cost!