Certainly no more badass than Mad jack.
😮
Using a Google Reader as a lifestream archive
After I set up my lifestream to track a host of custom actions (like comments on other blogs, or votes on FSD) I discovered that many of the feeds I was using were having a item limit (from 10 to 20). That meant that if I made, say, 11 or more comments in various other blogs, the older ones would dissapear from the list, even if the date was still in the limit I’ve set. Only the newer 10 would always be shown and it would seem as If I made no comments below that date.
It was not a bigg issue but I really wanted it to work right so as to get a more accurate overall idea of where I am spending my time (graphs and so).
I remembered then, that Google Reader seems to always have the option to go back into older posts in the feed, up to the time that you made the subscription through it. This meant that it archives everything in that feed and stores it in its own server (Damn that Google storage is endless). Also, Google Reader has the option to put various subscriptions under a specific folder/tag, allowing you to read all the subscriptions together.
It also allows you to make that tag public. A Public tag allows anyone to see the feeds that you share through it but also gives you a feed to subscribe in turn. This feed, unlike the original ones, can provide feeds data as far back as the time you made the Google Reader subscription (and even further if someone else had subscribed to it before you, I believe). All I needed to to is append ?n=# at the end of my google reader feed where # is the number of items I want to see each time the feed is polled. (I selected 100)
So there I had it. An easy way to make an arvhive of my tracked activities without setting up my own database.
The only problem now was to make the lifestream plugin recognise from which feed in turn each item comes and then assign it the correct class. This was a more challenging question as my previous method to compare the original feed with the current item’s could not work. I discovered that I could filter by the items target link but since I wanted to use the same service to track different activites, it would always filter them the same way. In case you’re wondering, I use Getboo to bookmark pages that I commented on as well as FSD articles I’ve voted up, down or commented (Since, unfortunately, FSD does not provide such feeds by itself until now). The number of things I track through Getboo will only increase as I find more actions that don’t provide feeds.
Thus I needed some way to the original feed through my Google Reader lifestream feed.
Well, how very fortunate of me that Google Reader publishes an Atom feed and not only that but it includes the source of the original feed for your convenience. 😀
A little trial & error with the (truly excellent) SimplePie APIs, and I’d managed to find and write the code that I needed to use.
if ($feedurl == $greader_lifestream_feed) {
if ($source = $item->get_source()) {$source_url = $source->get_permalink();}
if (stripos($source_url, 'twitter') !== false) {
$class = 'twitter';
$tf_counter = $tf_counter+1;
$current_title = 'Status Update (Twitter)';
}
if (stripos($source_url, 'mycomments') !== false) {
$class = 'comments';
$extra_text = 'Commented on: ';
$getboo_comment_feed_counter = $getboo_comment_feed_counter+1;
$current_title = 'Comments Elsewhere';
}
if (stripos($source_url, 'fsd') !== false) {
$class = 'fsd';
$extra_text = 'Voted up: ';
$fsd_counter = $fsd_counter+1;
$current_title = 'Votes up on Free Software Daily Article';
}
}
And yes. I’m proud of my little code snippet. 😀
So now, if I have any feed with a sufficiently low item limit, all I need to do is subscribe to it through Google Reader and add the appropriate if statement in my plugin code and presto! Instant archiving. There is only a (very) slight problem in the sense that the lifestream update is delayed by the time it takes for google reader to refresh the original feed and then update the public tag.
All I need to do now is figure out a way to separate items in the google reader feed through the category tags that getboo provides. That would be sweet as it would cut down on line number but unfortunately the get_category() simplepie function does not seem to work as intended for me 🙁
How to hunt for WordPress performance hogs
So after my previous post on how I discovered my major causes of wordpress pain, I kept looking on what is causing my other delays. Eventually, while looking at my source code, I discovered that my wordpress page was always reporting how many SQL queries it took and how long they took to complete. Looking at the footer, this was done by this command:
<?php echo get_num_queries(); ?> queries. <?php timer_stop(1); ?> seconds
Now, I understood that this was very helpful but I couldn’t understand why it was returning a time like 10 seconds when the page was loading for 4-5. A quick google search for this code however did lead me to a page that was explaining how to Keep WordPress overhead down. Through there, I realized that those seconds were some sort of CPU time taken to draw those queries.
Through this then, I now had a semi-accurate way to monitor the impact of each plugin on my site. I only needed to figure out which parts of the site were hogging down the queries and increasing the load time.
I created a quick table in a Spreadsheet and started logging down the differences.
- First, I made the performance text visible, so that I don’t have to look at the source code each time (You can still see it by scrolling to the bottom of the page)
- I disabled the WP-Cache so that the queries are done every time
- I then started disabling each plugin in turn and reloading a page. Then I compared that with the previous loading time (where the plugin was enabled.) I noted down the difference in queries and the average/approximate CPU time difference.
Unfortunately since the CPU time tended to vary from load to load, I couldn’t get an accurate number of the difference. The table I made in the end looked something like this
| Plugin name | Queries | CPU Time Taken |
| Bad Behaviour | 1 | 5 |
| Hide Text | 0 | Negligible |
| Greeklish | 0 | Negligible |
| One click install | 0 | Negligible |
| SimplePie WordPress Plugin | 4 | Negligible |
| Security Scan | 1 | 1 |
After I went through all the plugins, then I started doing the same thing with my various widgets and snippets of code in my theme that might be causing this.
Once this was done, I had one likely suspect plugin and a few widgets that were probably contributing to this slowdown. You see, my theme, a very heavily hacked HemingwayEx, was using some custom widgets in order to draw from WordPress the latest comments, recent posts and whatnot. These widgets queried the database each and every time a page would load which was quite frankly impractical.
Fortunately a short time before, I had discovered the very useful bundle of post plugins. They were also there to hunt for recent comments & posts and related posts and they also included a built-in caching mechanism! As sweet as it gets.
I quickly modified my widgets to run this code instead and the results were wonderful. By disabling also the single other plugin I found that was slowing things down, Bad Behaviour, I’ve now managed to drop the loading time considerably (or so it seems for the time being). Not only that, but I’ve managed to put a few new toys to use that do a better job than HemingwayEx’s built in functionalities.
Currently my site seems to load lightning fast compared to before but I’m not absolutely convinced this will not change once my shared hosting environment gets bogged down. Hopefully, even if that happens I will still have a much much better performance than before.
I am still stuck at around 65-80 queries per page load but I can’t figure out where they come from. I am hoping that I’ll eventually be able to trace it but for now it seems enough.
Btw, I just loved the functionality of the Post Plugins to measure their own execution time and report it back to you. This is something that all plugins should be doing in order to give the web designer an idea on what is going on with his site. I’m going to work to see if I can implement that in the Lifestream plugin I’m working on and then see if I can place it on other plugin as well.
Btw (2), Can anyone explain why more people don’t take advantage of the Plugin cache plugin? I am fairly certain that a number of heavy plugins (like Popularity contest) could make good use of it.
As always, if you have any more ideas on where to look and tweak to improve my WP performance even more, I’ll be glad to hear them (AKA Whining because no one is commenting on my blog :P).
PS: It’s a pity that I had to disable Bad Behaviour as it worked quite well until now. I also don’t like to reply too much on just a single anti-spam plugin. Unfortunately, the performance hit was undeniable and I’d rather my site doesn’t take an extra 3-4 seconds to load every time.
Squashing Wodrpess Performance Bugs
I think I’ve finally managed to trace down and smite all the things that were making my site so Gawd awfully slow. I had finally been annoyed enough by 30 sec loading times that I just had to do something about it.
Initially I thought it was with my host, and I did contact support about it via Email. They did check out the MySql slow logs and noticed some strange queries from BadBehaviour, but when this little plugin blocks around 1000 bots per week, it’s not worthwhile to disable it :).
In Dreamhosts’s opinion I should try and disable my plugins and then switch to another theme to check. Well, even though it was painful, I did it but it didn’t really make much of a difference. My site still seemed to be slow. However a test site I created had a quite normal speed. Perplexing
Initially I thought that it might just be the difference in DB size and indeed, my other WP sites, which have much less content that this, seemed to be loading faster (albeit slightly).
Fortunately a few google searches led me to some good content. Initally I discovered Firebug from a post explaining how it can help you diagnose WordPress performance issues. That was quite helpful to tell the truth. I managed to immediately spot that the WordPress Automatic Upgrade (WPAU) and the Popularity Contest were severely slowing down my Dashboard. I also found a few items in my theme that were just taking too long to load (like my FSF badge for some reason). Disabling them dropped the load time but I still had a problem.
You see, for some reason there is a 5-9 second loading time from the time I click on a link, to the time the page refreshes and starts to load. Firebug, unfortunately, will not give you any information on what is causing this, other than to tell you that the link is loading. It will not even take into account this initial loading time when giving you the total load time of the page (thus a page taking 13 secs will be reported as taking 4.5). Since this loading time dissapeared
This meant that I had no idea what was causing it, so back to the search I went. This time I managed to find a more interesting post on Improving WordPress performance which not only gave some great info, it linked to some other sources. Unfortunately, most of the tips given were to be used on a private server and were not really applicable to a shared hosting service like mine. Of course, this person was talking about receiving 500.000 hits a day, whereas I’m barely receiving 3000…a month. Still I applied the wp_config.php edit although it didn’t seem to make much of a difference.
My final hint came from wolfie who advised that I remove all widgets and links that point to other addresses as these may increase the loading time. I did that and I also took down the Popularity contest as well, just to see the results.
It seems it worked. The initial loading time has dropped to around 4-5 seconds. Although I don’t have as much bling in my site anymore, I am hoping that the increase in speed will make up for it. Unfortunately 4-5 secs is still quite bad and while I still don’t have any idea where to look in order to fix that, it’s at least better than 6-9 secs + 4 secs of extra stuff. With the WP-Cached page, the load seems to be instantanteous OTOH so at least that helps. Still for a ~100 hits per day, it certainly should not happen.
As an aftereffect, I decided to finally update this site to WP 2.5 as the functionality of Popularity contest was the only thing preventing it. At least I now have a snazzier Dashboard 🙂
After improving my performance I decided to also start a WordPress Performance page on the dreamhost service wiki. I would urge all of you, Dreamhost customers or not, to take a look and improve this page so that others that have similar issues can find a focal point to troubleshoot. I also hope to see tips specific for Dreamhost shared hosting customers eventually.
If there is a good WordPress improvement page or tips that I should check btw, feel free to let me know. Also, if you have any idea what causes these ~4 secs of initial loading delay, I would be really glad to hear it.
Semi-Maintainance mode
Bw aware that I’ll be trying to figure out where my performance problems are coming to so you might see various things not working.
Charted life
I’m not the one to pat himself on the back for the smallest thing, but I must say that I feed quite proud about my latest little hack in my lifestream.
I was initially inspired by yahooza’s lifestream (via the lifestream blog) after I saw that little chart on the right side. He used the google charts API to achieve which seems quite easy to use and very impressive for online use. Thus I digged in my Simplelife plugin and started hacking around.
You can see the result below. For my limited php knowledge, this was quite a achievement 🙂
Of course the above is a static image, but within my lifestream page, this is updated dynamically every time the page loads. I like 😀
Of course, this will soon be available to you all since the Simplelife v1.4 is coming out soon and the author would like to implement some of my ideas in it.
Now all I need to do is to find a way to get the dynamic image in a format that I can put in other places without having to run the script along with it 😀
Where're my damn icons?
I don’t get it. My lifestream icons are not loading for some reason even though you will see in the page source that they are loaded as part of the class. For some reason, initially the last.fm and facebook icons stopped loading after I hacked the SimpleLife plugin to make it play with all kinds of feeds. Then, I discovered why the colours of the custom feeds were not being used (a missing colon) and fixed it, and now <a>all</a> my icons are gone…
Grrr!
Nothing I do seems to fix this and I just cannot see why it is happening. There does not seem to be an obvious error in the code and the rest of the css class (text, background colours) are loading just fine. In the source of the page, you can see the css clearly loading correclty
a.lastfm {
background: 0c0c0c url(http://dbzer0.com/wp-content/plugins/simplelife/lastfm.png) no-repeat 10px 50% !important;
border-top: 1px solid 0c0c0c !important;
border-bottom: 1px solid 0c0c0c !important;
color: #BFBFBF !important;
}
And then the item call
<li class=”item”><a href=”http://www.last.fm/music/Machinae+Supremacy/_/Stand” class=”lastfm” title=”db0’s Recently Played Tracks”><span class=”timesf”>09:48</span> Machinae Supremacy – Stand</a></li>
And the icon exists…
Anyone have any idea what is going on?
At least now I can have feeds from any source (feedburner, del.icio.us, google reader) without losing formatting. I just need to wait for the next plugin version to add options for more streams 🙂
UPDATE: Nevermind. I found the culprit. Apparently I’m blind…
Mighty plugin hacking
Well…for my newbie standards at least 😛
I’ve been playing around with the SimpleLife WordPress plugin, trying to create a simple lifestream I can embed in my about page and perhaps in my sidebar as well. It turned out that this was a job which needed a bit more attempt to make it work.
Below are the changes I did in order to wrangle it.
- Had to edit the plugin in order to get it to diplay correctly in the plugins page. For some reason it floods the screen with it’s contents. Apparently editing and saving should solve it but in my case I needed to delete some newlines as well.
- Changed the classes .date and .time to .datesf and .timesf and modified where they were called. Leaving them as they were, they were screwing with other css classes on my site under the same name
- Changed the if statement for last.fm to look for the string ‘last.fm‘ instead of ‘last‘ since that string may be a part of any other url.
- Put three new if statements, similar to the ones that check for last.fm and facebook, that check if the link goes to this blog, the ACP or the Wesnoth Journals. Then set the class according to that. The way the plugin is setup, it does not assign the class depending on the feed url bur rather according to the current link url. This means that you cannot use feedburner as your feeds address as it will never trigger an if statement[1]. I actully find the way this is handled a bit weird as it would serve much better to check the feed url and assign a class, rather than check the current link. I’ll have to check if that’s actually feasible…
- Commented away the if statements for the first three feeds. I’m using the custom ones instead now.
- Manually set the colour of the last two feeds within the plugin. For some reason even though the php call takes the variable I’ve set in the plugin configuration page (I can see it in the source code of the page) the colour is not being used and the text stays black. Weirdly enough, the background colour is changed well enough and the “blog feed” works just fine.
- I didn’t want all my delicious feeds to be posted because I seem to be doing an awful lot of them. I prefer to log only the ones that signify my comments for which I use the mycomments tag. Fortunately I’ve discovered that you can use a del.icio.us tag by using the form: username/tagname
- Created a new page template and inserted the php code there.
All-in-all, the plugin seems very promising and it already seems to work fine for me. However it does seem quite error-prone and lots of people most likely won’t be able to use it yet. Fortunately this will be fixed in the short future.
Now I just have to see if I can add more stuff for it to track.
Liars
Unfortunately it seems that creationists are not the only ones limited to lying. There are many others who even though they claim an intellectual base (in biology no-less) but are just as shallow and quick to turn to comment deleting and banning in order to silence and the ones that refute them.
Thus, witness the my recent exchange with one such liar with whom I tried to engage in a conversation after I read a recent post from the Black Sun journal. Initially I left a simple comment and didn’t expect to respond any more. However after taking a second look and seeing that I’ve been labeled as an Objectivist (of all things)I decided to leave a second one.
I expected it to be deleted in short order which is why I kept my browser window open to it. And this is exactly what happened. Not only that, but the blog author decided to tell blatant lies in order to save whatever credence she had left with her audience. Unfrotunately, for some reason my browser window reloaded and I lost my comment (although you can see when I tagged it here) which means I cannot easily copy-paste it here.
Within my reply, in short, I tried to actually discuss the matter with her. I explained that she was doing the all too common fallacy of appealing to emotion. I explained how having the goverment take an active hand in the curriculum does not lead to totalitarianism, as exemplified by the situation in Europe where not only is the education level superior to the USA but homeschooling is also illegal. I asked her to put aside for a moment the communism/fascism idea and actually try to discuss the issue at hand.
What did I get for my trouble? I am being labeled as Black Sun’s alias, an Objectivist and a Troll.
Now, you would assume that anyone with even 2 minutes to space would very easily discover the validity of the first two claims by visiting my blogger profile, clicking on the “My Website” link and then coming to the Division by Zer0. Then it would be obvious that I am independent from BSJ and also find out my thoughts on objectivism (Hint: They’re not positive). It would also be easy to surmise that I am also not a Troll, unless Troll for this particular hypocrite includes anyone who disagrees with her…
But no. Even that simple task was too much to ask for this Homeschooler. It was just so much easier to delete my comment, put her comments on moderation (so that we don’t spoil her party) and put invalid labels on me and everyone else who dissagreed with her. Just read Black Sun’s latest post for more amusement.
If this is the kind of discussion she is having, I’m feeling sorry for her kid(s). I really am. Here is a child who will grow up learning that you should never challenge your authority figures or else you risk losing your rights to speech. And Gawd help the kid if he so much as dares to say that Communism/Socialism has some nice ideas. Seeing how she reacted to people proposing goverment intervention in order to save children from being kept ignorant, I’m half-expecting her to explode and start urgent brainwashing procedures (Yeah right, as if she’s not doing so now…)
It’s impressive though. Even Objectivists were not so rude so as to delete comments without a fair warning and an attempt to discussion.
Unfortunately people like this seem to be perfectly happy to live in their little bubble world where all they hear is praises from their friends. Putting their head in the sand is apparently a very appropriate method of dealing with issues and they’re displaying the classic “Live and let die” mind frame that has, and still is, creating so many problems in the world. It is a pity but like a hedgehog, when reality rears its ugly head, they will prefer to curl up in a ball rather than face it.
I have no specific problem with that when it affects only themselves but unfortunately not only do they brainwash their children to act in such a manner but they are also spreading their lies to the blogosphere without havign to deal with any feedback. Against such action, like Alonzo Fyfe says, our only course of action is Words and Private Actions. You have seen my own actions already and these are to spread the word and label them as appropriate.
I urge any and all of you (yes, all five) to expose such people for what they are by linking to them with keywords that describe them. I selected “Liar” in this caseand Black Sun selected the quite appropriate “Hypocrite”. They may be able to delete comments but unfortunately for them, they cannot delete linkbacks on the web. Hopefully anyone who is looking at whom is linking to them is bound to discover the antilogue…
If you dissagree with my action on the other hand, I’d love to hear your opinion on this.