Cloudflare Automatic IP Failover

Update: Starting 9th November 2016, Cloudflare will no longer support API v1 which we’ve used in this post. You may look for migrating to v4 and example scripts such as lyoshenka’s great work ūüėČ
Howver you can read our explaination below to figure out how we achieve Failover using CloudFlare service

We use Cloudflare for our service and we use it to have server switchover or failover in case our main server go down, and it been working perfectly ever since!

Wouldn’t it be perfect to have a DNS failover, saving costs by avoiding a hardware and also reduce risk of having main load balancer go down?¬†It is possible to have DNS Load balancing by having two IP Addresses point to same record A, but we go for it in another article.

In this article we will cover how to have CloudFlare perform automatic IP failover for your servers, so in case main server go down, backup server kick in.

Please note for this to work seamlessly, you need your backup server be exactly same as main server. You can achieve this by having MySQL replicated and file system synced using lsyncd. You can also have your backup server show a temporarily message.

If you were to use a real DNS to set IP, it would take hours for new IP to apply to DNS servers around globe, and your visitors would still get pointed to crashed server but not with Cloudflare. new DNS will kick in less than few minutes (1min approx, and max 5mins)

You will need to run monit server in your backup server, ideally you can run monit on a separate server which only have monit server on it. You can get a simple and budget friendly vps linux cloud server from vpsdime and have it check status of your main server.

Let’s cut it short and get to instructions!

Please get your API key from Cloudflare. You can get it from Account tab in cloudflare. Cloudflare is free!

Then you need to shell scripts, one to change IP to backup, and one to change IP to main. Here we have provided both scripts for you. You can download them HERE and set appropriate configs into both files (IP addresses, cloudflare key, email address…)

You need to upload files in your backup server, in this example we use folder /hitsteps/cloudflare/

type in ssh console:

mkdir /hitsteps/
mkdir /hitsteps/cloudflare/

and then upload configured files there.
Make sure you give execute permission to this two files:

chmod +x /hitsteps/cloudflare/*.sh

Now, you need to install monit in your backup server:

yum install monit (or apt-get install monit)

and configure your monit to check your main server status by editing /etc/monit or /etc/monitrc and add following lines to end of it, edit 0.0.0.0 with your main server IP address and change alert email address to your address:

check host 0.0.0.0 with address 0.0.0.0
alert your@email.com
if failed icmp type echo count 3 with timeout 3 seconds for 2 cycles then exec "/bin/bash -c /hitsteps/cloudflare/tobackup.sh"
else if succeeded for 2 cycles then exec "/bin/bash -c /hitsteps/cloudflare/tomain.sh"
if failed url http://0.0.0.0/ with timeout 20 seconds and retry 2 for 2 cycles then exec "/bin/bash -c /hitsteps/cloudflare/tobackup.sh"
#else if succeeded for 2 cycles then exec "/bin/bash -c /hitsteps/cloudflare/tomain.sh"

Now you need to restart monit, and make sure it run on startup:

service monit restart
chkconfig monit on

Guess what? It is all ready! You’ve just setup an auto IP Fail Over¬†using Cloudflare.

Now, monit will check status of your main server by pinging it, if ping failed for 3 times on 3 cycles, then it will fire tobackup.sh file, same will happen if it cannot access webserver for 2 times.
Once server is accessible, it will run tomain.sh.

tobackup.sh will connect to cloudflare API and change your domain IP addresses to backup server. this change usually apply in less than 1 minute by our experience. monit will keep monitor your main server status and change IP back once it is accessible again.

As a sidenote, your cloudflare folder which contain scripts should have write permission because it will write logs into it. you also need to have following apps installed in your linux box for tomain.sh and tobackup.sh parse cloudflare API correctly: bc, cut, curl, sed, head, python

yum install python bc cut curl sed head

Do you have any suggestion to enhance this script? or any feedback? Please do let us know!

How fast are SSD servers? Performance comparison between HDD vs SSD

SSD make difference!

We’ve been hearing that SSD hard disks are new generation and therefor, all database intensive servers have to move to new SSD servers.¬†We gave it a shot and we never regret it!

In hitsteps, we analyze thousands of hits per seconds. this hits then categorized and processed by a background processor, and archived somewhere along with millions of other hits!

Given searching a visitor or looking back into archive of a visitor is an essential feature of hitsteps, we always had to struggle with performance of database searches, specially when searching for some visitors who visited months ago.

MySQL databases need RAM to hold all indexes of data, they also need a fast disk in case data are not in RAM. so we did the test in a mirrored server:

Our search parameter was a simple query of “Searching for a visitor for who visited hitsteps dashboard who are from¬†Canada since 6¬†months ago”

HDD server took 45 seconds to return query (almost near to timeout in most apache web server setups). SSD server took only 6 seconds to return query. Going to page number 2 in HDD took 15 seconds, in SSD just a wee 2 seconds.

It worth mentioning all other simple queries take less than 0.2 seconds in SSD servers and 0.5 in HDD servers.

Please do note that this times are closed source and relative to our script and codes. It might be different with other Applications, codes and scripts you might use in your server. RAM also play a major role here which is more appearance if your server is using HDD.

Nous parlons francais!

Bonjour!

It’s been a while we are working on this one, french language is hard to learn, and we now realized, it is¬†also hard to translate! Took us 1 month and half to finally get language ready all over hitsteps, but it come just on time for this new year, we welcome 2016 with a new language!

If you are a french user and you have selected french as your language during signup, hitsteps now automatically show french language to you,

New users will be detected by their geolocation and correct language will be set for them, so everything goes smooth and automatic.

If you would like to get back to english language, you can do so by using language switcher at footer of our homepage.

There are lot of languages out there, but it take very long time to translate for each of them, therefor we will plan a community driven system where everybody can collaborate their own language and shape hitsteps for their own, wait for that!

How Visitor Profile unifying works in Hitsteps Web Analytics?

We are happy to announce a new feature, called “Cross-Device” or “Visitor Merging”, you get the idea.

You can see full customer journey in your website. So your users can access your services on their phones, tablets and desktop. Hitsteps now will keep all of their different devices profile in one unique profile, making it easy for you to keep track of each particular users.

Basically, what it do is to assign an Unique ID to your visitors (say their email) once they leave a comment or once they login.

Next time same user come to your website, even with another device, mobile or tablet, once they login back, Hitsteps will detect their previous profile and merge them together. so that new session will be continue of user’s previous session. be 1 day before or 1 month before.

You can register your hitsteps account now and give it a try. If you use wordpress, be sure to use our WordPress plugin to automatically integrate this feature.

It comes handy when your users register on your website from mobile device, then few days later continue their browsing from a Desktop and purchase something from you. If profiles were to kept separated, you look at profile who purchases from you and you have no idea how this visitor even find your website! Worry no more!

Our WordPress users can update their WordPress plugin to v4.81 now and benefit from this feature on their WordPress based website (WooCommerce and such)

Unified visitor sessions

Here is how it looks like in your visitor profile reporting:

visitor profile timeline

Our other users will need a little bit of coding.

Just put following code above hitstep’s tracking script, you need to define a unique ID using server-side scripting, and we will take care of rest:

<script>
_hs_uniqueid=”YOUR_DEFINED_UNIQUE_ID”;
</script>

so that you can define a unique ID for each user, say their email address or their User ID and have it linked to visitor who is browsing your website.

While you are there, don’t forget you can set a name for your visitor using ipname parameter as well:

<script>
ipname=”YOUR_VISITOR_NAME”;
</script>

Once a user login and If we have user’s unique ID stored before, All events he fired and pages he viewed perior to login, will be linked to main profile ID by Hitsteps Analytics and make a unified timeline of full user behaviour.

Easy way to Track visitors on Shopify Store

Since we have launched Hitsteps Shopify App last week, we have applied many minor changes to perfect it.

It is a deal breaker for your business and make you be able to understand your visitors in deeper depth.

Here is how to install Histeps Web Analytics on your Shopify:

  1. Open Hitsteps App in Shopify here.
  2. Click on Get to get the app for your store.-2015-10-23_20-43-54
  3. If Shopify Asked you to enter your shop info, enter it. Otherwise go to next step.2015-10-23_20-44-15
  4. Shopify need to ask for Install confirmation. Please Install hitsteps shopify analytics.
  5. You’ll need to create your Hitsteps account (or login if you already have an account in hitsteps) Just fill up the form.2015-10-23_20-41-43
  6. Enjoy looking at flow of your visitors in your newly created hitsteps analytics dashboard.dashboardl

How to install Hitsteps Web Analytics in Shopify

hitsteps love shopify

We are proud to announce hitsteps full integration with Shopify ecommerce and online shop service.

If you have a shopify shop, don’t worry about changing themes and inject hitsteps code. Simply install hitsteps app from the Shopify App Store.

The app will automatically install, add your website to hitsteps, create predefined shopify labels for you to track sales and products, and automatically install hitsteps code on all pages of your Shopify Store.

 

Enforce Anti-bot & Anti-spam analytics filtering

Today we have a real deal for you, for those of you who are tired of getting excited by a high peak on your traffic only to know that it’s been a Microsoft Bingbot, Google bot, Facebook crawler or any other bot crawling around in your website.

We had a basic bot filtering in place since our launch last year, but now we have tighten this system.

Now we actively scan each one of your visitors to make sure we are only providing you with real human analytics who are accessing your website.

It helps to give you a real analytics insight, purely based on real human visitors and avoid fake generated pageviews from various web services.

For example, if somebody share your page on facebook, facebook will open your website to take a snapshot for thumbnail photos. this process will repeat for each new shares. Normally you might think you are getting lot of visitors, but in fact they are all Facebook thumbnail generator bots trying to open your website and get list of photos to show them upon sharing. It give you false hope and screw your analytics data. We have eliminated it!

As a result of filtering this visitors from your analytics, Hitsteps might show slightly lower hits compared to other analytics services where they count all hits. It come to our attention that many many analytics still track visitors like in ancient days and ignoring the fact that nowadays, many web services have a tool to scan your website and thus, they show you not so useful analytics insight.

If you were still unsure about hitsteps analytics, but now you made up your mind, go ahead! We will meet you inside. Get Hitsteps WordPress plugin.

Percentage values in Email report and Dashboard pages

We have decided to change the way we shown percentage values in hitsteps.

You most probably seen that first item ( e.g. in popular pages section ) is always 100%. Reason behind it was that we calculate percentage value relative to highest hit. We did this back in our private beta times when our customers wanted more contrast between each value.

After some internal discussion and our customers feedbacks, we have decided to change percentage values to be relative to total hits, as a result, represent a better picture of each row.

This change has been applied to your email report and dashboard as of today.

Continue reading Percentage values in Email report and Dashboard pages

Referral Spammers, no more!

If you looked at your analytics data recently and looked at your referral section, probably you have noticed some strange referrals from websites that you cannot identify why they should send you visitors!

email-gateway-anti-spam

We are strictly against spam and automated bots in hitsteps and we are against this method of referral spamming too. as you know, your analytics number and list of online users only reflect your true human visitors and you won’t see bots, spammers and spiders in your web statistics.

These services are not actually sending you real traffic! we have analyzed all traffics coming from them for all of our customers and we noticed a bounce rate of 100%, it means all visitors come through them close website as soon as it is loaded. this is highly unlikely scenario from human-line browsing.

so why they do this? They are actually advertising to you. this way, they can show their name to you in your statistics report. so you get curious and follow up to see who are they and visit their website! so this way they increase their visibility not only in hitsteps, all other analytics such as google analytics, piwik and … too, as they show them as your top referral once they have send enough (useless and bounced!) traffic to you.

We don’t know how they send this traffic to you, but we assure you this traffics are spam and therefore their name in your referral section is spam as well, as a result we have took necessary action to protect our users from spam statistics and continue to provide real human web analytics.

Starting from today, this referrals won’t show up as your online visitor, won’t count as a pageview and won’t show up in your referrals in hitsteps.

If you are only using hitsteps, then you are protected!

Make widgets in your plugin compatible for WordPress v4.3

WordPress v4.3 is just a week and half away to be released and it is a major update for developers!

WordPress team being innovative again and deprecated PHP4 style classes in old WordPress, therefor, your plugin is going to show an ugly error in case you use WP_Widget class to show a widget in your website sidebar.

We use hitsteps live support widget and statistics widget in our plugin, so we had to update our codes too and…

Here is what to do!

Open your editor and search for any reference to “extends WP_Widget”

2015-08-09_01-18-49

Your widget class have a name, and most probably (PHP4 style), name of your plugin is used as a function inside of same class. Look here, here’s ours:

2015-08-09_01-22-06

Now, change function name to __construct(). this is new way of defining default function of a class.

And inside of that function, you’ll have a code that call WP_Widget.
It can be¬†parent::WP_Widget, $this->WP_Widget, {something}->WP_Widget… which all are deprecated!

You have to change that to parent::__construct(…);

Here is how it should look like after you done all this changes:

2015-08-09_01-26-15

Number of¬†“extends WP_Widget” lines you have match number of widgets your plugin create. You’ll need to repeat this process for each of them.

Congrats! Your plugin is officially compatible with WordPress v4.3

If you wish to read more about this, you can read this and this from official WordPress team.

We have just released our update for WordPress v4.3 users today, so you can expect a smooth upgrade to new WordPress regarding our analytics service ūüôā