You set everything up to scrape some serious data, and you’re excited about the information that you’re going to get. All you need to do is put your web proxies and your computer on autopilot, and you’ll be good to go. Then, you can come back the next day, and your computer will be full of all of the info you need.
It’s as simple as that.
Only, there’s a problem when you come back. For some reason, your proxies stopped working. Everything was fine when you left, but something happened when you were gone. Fortunately, you probably don’t have a serious problem on your hands. There’s a good chance you have a common issue that can be fixed relatively quickly. Just try some different troubleshooting tips, and your proxies should be up and running again in no time at all. Then, you can put your computer and proxies back on autopilot and go back to living your life.
More times than not, proxies come with an authentication system that you can have to use to access them. You have to input your credentials, or you won’t be able to access the web with your proxies. This is a safety protocol and very important.
You’re probably thinking, “No problem. I can put a username and password into the system. I have that covered.”
That’s true. It’s really easy to put a username and password into the system, but things get a little more complicated when you use some software. Some applications don’t have spots for you to put your authentication information, but it’s still required.
Let’s say, for instance, you use IgersLike. This is a really cool social media tool. You can use it to create and manage multiple social media accounts. You need a proxy to do that properly, so it’s really popular for people who use proxies. However, it doesn’t have an authentication system built into the software. That means you have to contact your proxy provider so you can use the application. Otherwise, your proxy will stop working when you use IgersLike.
If that seems like too much trouble, stick with tools like ScrapeBox and GSA that have authentication areas built into the interface. Then, you can plug your information in and leave the house without worrying about your web proxies working.
It’s nice to set up a proxy and tool, and then leave the house. You know that everything is running on its own so you can go and do your own thing. Unfortunately, you might get carried away and turn your settings on too high. Then, your tools will act like bots, and the websites will catch on. When that happens, your proxies will get banned.
It all comes down to two settings, which are your frequency pings and your threads.
Your frequency pings refer to the number of requests that you send. You don’t want to send too many requests at once or it will look suspicious. While many people like to set a low delay between pings, you should have a minimum of 10 seconds between pings. Anything lower than that will set off red flags. In reality, you should put at least 14 seconds between pings just to be safe. That extra 4 seconds isn’t going to hurt you when it comes to getting information, but it will help you hide from site administrators.
Next, you have to worry about the threads. Threads refer to open connections. The more open connections you have, the faster you can pull data, so you might feel compelled to have hundreds of open threads at once. As nice as that sounds, it’s a huge mistake. Limit yourself to no more than one thread for every 20 proxies. You can do even better by limiting yourself to one open thread for every 10 proxies. While some people say that you should have a 1:1 ratio, you don’t have to be that careful. Websites aren’t that good at tracking open threads and proxies. If you stick to one thread for every 10 to 20 proxies, you’ll be fine. Websites shouldn’t detect any bot behavior.
Check Your Internet Connection
When you act like a bot, other websites don’t just take offense to your behavior. Your ISP might also notice and take action. This often happens to inexperienced web proxy users all the time. They get their proxies and software and hit the internet full force. They start scraping data as fast as possible, and their ISPs wonder what is going on. They suspend their internet connections because they are worried that some sort of attack is taking place. If your proxies aren’t working, see if this has happened to you. Log out of your proxy and see if you can use the internet without the proxy. If you can’t, you have an internet problem.
Fortunately, you typically don’t have to call your ISP to clear this up. You just need to follow a couple of steps to get (and stay) online.
First, you need to reset your internet connection. Unplug your modem, load everything up, and get back online without your proxy. Log back in and make sure you can use the internet. Then, get online with your proxy. Make sure that you can visit various websites with your proxy. If you can, you are free to use your proxy.
Now comes the second step of the process. Don’t do anything to make your ISP suspicious again. You have an aggressive ISP. If you do anything to stand out, it will take you offline again. However, the next time, you might get banned instead of just taken offline for a short period of time. Then, it will be a lot harder to get back online. Make sure you don’t do anything that sends up any red flags, or you might pay a steep price.
Check Your Firewall
Your firewall does a great job of keeping malicious software off your computer. It also does a bad job when it comes to letting you use your proxies. You might turn your firewall off so you can use your proxies, but sometimes, firewalls have a way of turning back on when you aren’t around. If that happens, your web proxies will stop working.
Your proxies need access to open ports to work, and firewalls deny that access. If your proxies aren’t working, look for firewalls. If they’re on, turn them off and try the proxies again. Often, your problem will be fixed immediately. Your proxies will start working like nothing happened.
If your proxies still aren’t working, check your router. Routers often have their own firewalls, and those firewalls can prevent your proxies from working. Log into the settings and look for firewalls. Also, see if there are any ports that are blocked, and if there are, unblock them. In addition, see if your router has blocked the application that you’re using. This happens from time to time. For instance, some routers block ScrapeBox and you have to go into the settings and unblock it.
Once you go through the settings, check your proxy again and see if it works. If it does, you can set everything back up on autopilot and let it get back to work.
Check the Software and Site
It’s normal to think the problem is on your end. However, sometimes, your web proxies aren’t working because they can’t connect to the software or the website. Disconnect from the proxy and check the software and the website. If you still can’t connect, the problem isn’t with the proxy. It’s with the software or the website. All you can do is wait for the software or website to come back up.
However, if the problem with your software persists, you might need to call the manufacturer. You might be missing an update or you might be dealing with a compatibility issue that is preventing your software from working. The manufacturer should be able to take you through the process so you can get your software running again.
Make Sure It’s a Legitimate Proxy
Time and time again, people can’t access their proxies because they aren’t legitimate. This might seem like a head-scratcher. Why would someone give you access to a proxy that isn’t really a proxy? Let’s take a closer look at this.
First, it’s important to understand that proxies fall into two categories. You have public and private proxies. Private proxies are the proxies that you pay for. These are dedicated proxies, meaning that you get dedicated resources and your own IP addresses. You can also get semi-dedicated proxies, where you share IP addresses with a couple of other people. You still have ample resources at your disposal when you select semi-dedicated proxies.
Then, you have shared, or public, proxies. These proxies are generally free, but they come at a price. You share these proxies with anyone and everyone. Whenever someone logs into use one of these proxies, you have to share it. There are only so many resources to go around, so you might end up with a tiny piece of bandwidth when you use one of these proxies.
Sometimes, free proxies stop working because you only have a little bit of bandwidth, but more times than not, they stop working because they are just a front for hackers. They aren’t real proxies. Instead, hackers set them up so they can steal your information.
Hackers use these so-called proxies as bait. They put them online and wait for people to input their information. Sometimes, they don’t even need you to put information into the system. A simple connection with the server is all they need to gain access to your computer. Then, they can sniff around your computer and get all of the information they need. They can take your banking information, social security number, and more. If it is on your computer, they can snatch it. They can even leave software behind that continues to gather information. Then, that software can feed the information back to the hackers.
Hackers don’t just use the free proxies to steal identities. They also use them to run DDoS attacks and more. You might log onto a free proxy and become a part of a DDoS attack without even realizing it.
If you can’t figure out why a free proxy isn’t working, it might be because it was never meant to work in the first place. It was only meant to be a front to steal your identity to or to run an attack of some sort. That means you need to disconnect with the proxy and wipe your computer clean. There is a good chance that you have been hacked and you need to get rid of any viruses that you have. You also need to monitor your credit to make sure that you don’t end up with charges. It is a good idea to invest in a credit monitoring program if you think you have been a victim of one of these attacks. The program will let you know if there have been any changes to your account. That way, you can take action as soon as something happens on your account. You might even be able to close an account out before the hacker makes any charges.
Of course, the best thing to do is to stay away from these free proxies. It’s much better to pay a few bucks for a private proxy and maintain your good credit.
As you can see, you have lots of options if your proxies aren’t working. Go through the list and check everything so you can get your proxies back up and running quickly. Then, you’ll be able to scrape data or do anything else that you need to do without having to worry about your proxies. Once they are back up and running, they’ll be like a well-oiled machine, scraping data when you’re sleeping or out of the house.
The information contained within this article, including information posted by official staff, guest-submitted material, message board postings, or other third-party material is presented solely for the purposes of education and furtherance of the knowledge of the reader.
All trademarks used in this publication are hereby acknowledged as the property of their respective owners.