What do you do when you want to search and replace out everything BUT a certain phrase?
%s/^\(\(REGEX\)\@!.\)*$//g
And yes. I do blog these so I can find them easily later. Heh.
One of my favorite things about Vim is that it takes regular expressions by default when searching and replacing:
%s/[A-Za-z]* is going to the store/Bill is going to the store/g
One of the problems I’ve run into, though, is that Vim seemingly arbitrarily defaults certain characters to their literal meaning, rather than their regular expression meaning. A good example:
%s/[A-Za-z]* (is going to the store)/Bill \1/g
The above may look like “Frank is going to the store” will change to “Bill is going to the store” (because the \1 is Vim’s way of designating a group). But, in fact, the above will fail because it’s literally looking for the parenthesis… The only way the above would work is on “Frank (is going to the store)”. In order to use regular expression grouping , you have to escape the parenthesis in Vim:
%s/[A-Za-z]* \(is going to the store\)/Bill \1/g
There are several characters which don’t act as you might expect using regular expressions. Some of them include:
- ( )
- \
- |
- +
- =
- ?
If you want to use these regular expression characters, you must escape them before use. Example:
%s/Sean\|Bill \(is going to the store\)/Frank \1/g
The above example will find either “Sean is going to the store” or “Bill is going to the store” and change it to “Frank is going to the store”.
One of the biggest drags about developing for the web is the need to test your code in a multitude of different browsers and OSs. In my case, that’s meant having to have four or five different VMs that I have to run for testing. It’s been a pain, to say the least. Microsoft has a solution that gets you part of the way there — the “Microsoft Expression SuperPreview” tool. It’s a rendering tool that lets you quickly look at web design layouts in most of the different desktop browsers out there. it only renders, so it won’t replace functional testing in VMs (alas!), but what it can do is let you find simple layout and CSS issues long before you start your cross browser test suite.
Out of the box, the SuperPreview tool gives you local rendering engines for all IE distributions (6, 7, 8 and 9). If you’re interested in testing other browsers — like Chrome, Firefox, and Safari — you can go ahead and also sign up for the SuperPreview pay remote service. From my understanding, the local IE renderers will continue to stay free if you don’t subscribe to the remote service.
Renderers always have one serious limitation: because you can’t walk your functionality, dynamic content created from posted data isn’t rendered. SuperPreview solves some of that issue by providing a “page interaction mode” which allows you to use your installed browser to do things such as log in and navigate to dynamically created pages. Once you’re at the page you want, you can click “Preview” and it will display that page in SuperPreview. It doesn’t help you with Ajax type renders, but it’s still a pretty good step.
All in all, a nice tool. It solves about 50% of “only one Internet Explorer version can be installed on a single computer” issue.
I’ve been meaning for months to automate metrics functionality from Google Analytics, but have been totally put off by how complicated it seems to do something that should be pretty easy. In the end, I found what appeared to be three different ways of doing the same thing — which boiled down to using OAuth2. The biggest drag of using OAuth is that it isn’t crazy suitable for simple repeatedly-run scripts. It requires you to get permission from users — in this case, the Google user account you use to collect analytics — every time you start up a new session. In theory, this can be mitigated by using a daemon that maintains session information — then you’d only have to kick start it every time you boot. My specific needs were simpler than that though… I just wanted to fill up a spreadsheet and be done — and maybe do that every month or so. So here’s my quick solution that met my needs — I hope it can help others, since I had a tough time finding a good example.
First off, I’m trying my hand at Ruby because I need to rewrite one of my sites and I haven’t yet decided if I’m going to use Rails or Grails — Historically, I’m a Java guy, but I don’t know Grails all that well yet, so I’m evaluating Ruby as another possibility. So if you want to follow along with this post, you’ll want to make sure you have Ruby installed, along with the rubygems and oauth gems — they don’t come installed by default with the Ubuntu install of Ruby. I’m using Precise — your version of Rails might vary.
sudo apt-get install ruby-rails-2.3 sudo gem install rubygems sudo gem install oauth
After I install gems, I do a quick command line test to ensure that they’ve installed properly and that Ruby finds them in its INCLUDE path:
ruby -rubygems -e 'require "oauth"'
If it runs without error, you’re good to go. If you’ve got an error, check out your gem environment to try to debug:
gem environment
Once you’ve done all that, you’re ready to try your hand at getting analytics from Google. First off, you have to register your domain with Google. You can do so here — it’s not easy to find so bookmark its location if you have different sites you want to register. First, add the domain site that will be interacting with user accounts and click “Add domain”. You have to have a domain, even if you’re going to just run scripts on your development machine. Your domain will now be listed at the bottom of the screen. Click it and you’ll be asked to verify ownership of the domain. You can do this in a couple of different ways — such as adding an HTML tag to your home page or by adding a DNS record to your domain’s registration. When you’ve set that up, click the “Verify” button. Google will ask you for a callback URI — unnecessary in my case, but you might prefer to use Google’s API that way — and then will list out your OAuth Consumer Key and OAuth Consumer Secret. As the name implies, keep this information private or nefarious people can use it to trick users into thinking they’re you. The way things I read several examples, it seemed to me like I might have to actually be generating requests from the domain, but that wasn’t the case. I was able to run the script off of my development machine just fine.
Now that you have your key and secret, you’re ready to code a Ruby script. First — add your gems:
require 'rubygems' require 'oauth' require 'rexml/document'
I’ve referenced REXML here — it’s an XML processor for Ruby. Since Google feeds back much of its information in XML, you’ll need a way to extract the data you want. Next, you’ll need to create an OAuth consumer object. Replace out the words KEY and SECRET with the information provided by Google in the last step. Keep the paths as they’re listed — this pathing information isn’t about your site, it’s about how Google creates tokens.
con = OAuth::Consumer.new('KEY', 'SECRET',
{:site => 'https://www.google.com',
:request_token_path => '/accounts/OAuthGetRequestToken',
:access_token_path => '/accounts/OAuthGetAccessToken',
:authorize_path => '/accounts/OAuthAuthorizeToken'})
Next, use the consumer object to create a request token. Google will return back the URI that your users — in this case, the Google account that has access to analytics — use to give your application permission to access its information. Once I have the request token URI, I went ahead and printed it out on the screen so I could easily copy and paste it into my browser. When you navigate to the URI, Google will ask you to grant permission to the domain site and give you a permission key that you give back to the application that allows it to connect with Google. If I wanted to automate this script further, I could use CURL to complete the authorization for me — but that’s overkill for what I need to do. Instead, my script asks me for the code that Google generates and then gets an access token based on the completed request.
rt = con.get_request_token({}, {:scope => 'https://www.google.com/analytics/feeds'})
puts rt.authorize_url
puts 'What\'s the code?'
vt = gets.chomp
at = rt.get_access_token(:oauth_verifier => vt)
Now that I have my access token, I’m free to query Google for any analytics that the Google account is permitted to look at. I do that by getting information using Google’s web API and taking the results and dumping them into the previously mentioned REXML document object. I went ahead and created easy variables for my start and end date — the script is actually more complicated than what appears here, that’s why they’re actual dates — then I convert them to strings for the URI.
startdate = Date.new(2012,04,01)
enddate = Date.parse(2012,04,15)
doc = REXML::Document.new(at.get("https://www.google.com/analytics/feeds/data?ids=ga:ANALYTICS_ACCOUNT&start-date=" + startdate.to_s + "&end-date=" + enddate.to_s +
"&dimensions=ga:pagePath&metrics=ga:pageviews,ga:uniquePageviews,ga:bounces&prettyprint=true").body)
Once you have the information dumped back to you in XML, use XPATHs to navigate the DOM and get the information that you want. As an example:
doc.elements.each("//entry/dxp:dimension[@name='ga:pagePath']") do |ele|
pageviews += ele.parent.elements["dxp:metric[@name='ga:pageviews']"].attributes['value']
unique += ele.parent.elements["dxp:metric[@name='ga:uniquePageviews']"].attributes['value']
bounce += ele.parent.elements["dxp:metric[@name='ga:bounces']"].attributes['value']
end
Hope that helps folks out — I wasn’t able to find a lot of info out there on how to do simple analytics scripting without scraping.
Fujitsu scanners actually play pretty nicely with Linux, but the setup to work with gscan2pdf sure was a pain. For those that are interested:
- Install the Fujitsu scanner manager software on a windows box.
- Copy the 1300_0C26.nal and 300_0C00.nal files to a directory on your Linux box and make note of where you put them.
- If you’re using Ubuntu, open the /etc/sane.d/epjitsu.conf file. It looks something like this:
- Near the bottom of the file is the S1300. After ensuring that your scanner has been properly detected (by using lsusb) and matches the 0×04c5 0×11ed USB identifer, go ahead and comment out the line above it that says:
- I tried the 1300 driver first, but it didn’t work well for me. Since you don’t lose any functionality (that I know of) by using the 300 driver, I went ahead and used that. Of course, the S1300 drive might work great for you, so feel free and give it a try. Either way, point to the location you save the drivers to:
- Start gscan2pdf and scan a page. If you’re system was like mine, you’ll find that you get a bunch of gobbly-guk instead of a good scan. If that’s the case, you need to update the Sane perl module. There are patches listed on sites out there, but trying to figure out what to do with the patch is nearly impossible. Instead, make it easy on yourself by simply updating the Sane perl module via CPAN:
- If you get a CPAN build error complaining that it can’t find sane.h, be sure to install the libsane development files first:
$ cat /etc/sane.d/epjitsu.conf
# For scanners connected via USB on a known device (kernel driver):
#usb /dev/usb/scanner0
# For scanners connected via USB using vendor and device ids (libusb):
#usb VENDORID PRODUCTID
# NOTE: if you have to add your device here- please send the id and model
# to the author via email, so it can be included in next version. kitno455 at
# gmail dot com - with epjitsu in the subject line
# These devices require a firmware file in order to function, which must be
# extracted from the Fujitsu Windows driver. Presumably the Mac versions
# contain the firmware as well, but the author has no access such a machine.
# Firmware is installed in several different locations by the fujitsu software,
# using the windows 'search' feature to look for '*.nal' is the easiest way to
# find them. They should be ~65K, and have the scanner's name as part of the
# file name. They are often inside a .cab file.
# Copy the file someplace sane can reach it. Then update the line below.
# NOTE: the firmware line must occur BEFORE the usb line for your scanner
# Fujitsu fi-60F
firmware /usr/share/sane/epjitsu/60f_0A00.nal
usb 0x04c5 0x10c7
# Fujitsu S300
firmware /usr/share/sane/epjitsu/300_0C00.nal
usb 0x04c5 0x1156
# Fujitsu S300M
firmware /usr/share/sane/epjitsu/300M_0C00.nal
usb 0x04c5 0x117f
# Fujitsu S1300
#firmware /usr/share/sane/epjitsu/1300_0C26.nal
firmware /home/seank/Documents/admin/scansnap/300_0C00.nal
usb 0x04c5 0x11ed
firmware /usr/share/sane/epjitsu/1300_0C26.nal
firmware /your/location/300_0C00.nal
sudo cpan upgrade Sane
sudo apt-get install libsane-dev
There you have it! Enjoy your new scanner. It’s fast, well priced and has lots of features. I’m especially happy with its ability to scan both sides of a piece of paper at the same time.
If you’re interested in a Fujitsu ScanSnap S1300, you can use the link below to get to Amazon.com. The link may not show up if you’ve got a blocking plug-in installed — instead, you can click here.
After suffering a strange crash on Monday that caused me not to be able to load lightdm anymore, I decided to go ahead and upgrade to Ubuntu’s newest upcoming release, Precise Pangolin. I typically update during late beta anyways — so I avoid the “rush” when everyone wants to download on release day. I’ve used Precise for two days now… and the verdict is…
Meh. Which is a good thing. (See below.)
Frankly, not a whole lot seems changed. In fact, I’m a bit pressed to find one thing that I’ve actually noticed is different in Precise. Of course, there’s the stuff that’s been publicized — better and faster software center, improved unity start screen — but I tend to do most of my admin from the command line and I only use unity for launching. Strangely enough, the feature I was most looking forward to was the updated Displays functionality, which, in theory, would allow me to have different wallpapers on my two displays. But because I’m using the nVidia drivers with its special settings tool, the display tool simply sees my two displays as one big screen it calls “laptop”. Too bad for me.
That being sad, the system has been pretty stable for the most part. Every now and again, I’ll launch an app which obviously isn’t ready for Precise and get a bunch of “This app caused an error” messages all in a row followed by the system asking me to supply pertinent information to the developers. I’d be happy to participate there, but those two dialog boxes pop up six or seven times and I find myself clicking cancel over and over again. Hope the folks fix that before release. But errors don’t happen all that often and things have been pretty stable.
And here’s the “below” that I wanted you to look at. Am I disappointed in Precise? Not at all. Would I have upgraded from Oneiric knowing what I know now (without having lightdm forcing me)? Probably not. But that’s a sign, I think, of how far the Linux Desktop has come. To think that I’m now seeing Ubuntu much like I see Windows XP — an operating system that works great and doesn’t necessarily need to be updated — is a fantastic change for the better.
Just my two cents there.
PS: Yes, it’s been a while. I was blogging for a different network for a time, but I’m back to blogging for cmReady… so stay tuned!
Ubuntu no longer keeps versions of Sun Java in its multiverse repository. Instead, it it installs OpenJDK by default.
To install the latest version of Sun Java (currently 1.6), you first have to add the partner repository to the machine. Be sure to use the proper name of the distribution. In this case, “lucid” stands for Release 10.4.
Read the rest of this entry »
This is the third article in a series where I discuss how I dealt with the challenge of maintaining two separate but similar web sites in source control. This kind of scenario doesn’t really follow the Version->Maintenance->Next Version linear development process. Instead, it has common code and separate themes.
In article one, I talked about your choices in how to work with this kind of structure.
In article two, I talked about how to use the Convert extension to split the common code out of our database.
In this article, I’m going to talk about the third and least friendly choice:
Finally, if you believe that the common code will change infrequently, you can avoid site merges and instead use the hg export and import commands to merge only certain transactions to the other branch. This last choice may seem the most expedient – especially if you’ll have many small differences in the common code – but it’s also the most error prone. Should another developer accidentally do a push and merge, you’d better know how to recover from it.
I use Vim for pretty much everything these days. So I’ve decided to start a “tips” category for my favorite Vim tricks. Here’s a question for you: how can you paste into the command buffer? And more importantly, how can you copy the command buffer to the clipboard?
In a previous article, I discussed with you how I might deal with the challenge of maintaining two separate but similar web sites in a Mercurial source control system. These sites don’t really follow the Version->Maintenance->Next Version linear development process. Instead, they have common code and separate themes. I mentioned three options for dealing with this issue:
- Let’s say it’s only a few files that are different – what then? Perhaps the easiest way of handling this situation is to just maintain the second set of files in the same database. No need for branching!
- But what if I have a good portion of files that will need to differ? Say 100 or more? What then? The best option in that scenario would probably be to split the differing “theme” files out of the database completely using its Convert extension. Then I’d end up with three databases: the common code and two sets of differing “theme” code, one each for Site A and Site B. The upside is that future maintenance is painless and a big bonus is that creating a third site will be a breeze. The downside is that you need to know in advance what the differing code will be and, more importantly, you’ll have to rewrite much of the common code to allow for different entry points, parameters and scenarios. This is the most time consuming of our options but also the best option for future code stability.
- Finally, if you believe that the common code will change infrequently, you can avoid site merges and instead use the hg export and import commands to merge only certain transactions to the other branch. This last choice may seem the most expedient – especially if you’ll have many small differences in the common code – but it’s also the most error prone. Should another developer accidentally do a push and merge, you’d better know how to recover from it.
Today, I’m going to discuss the second case – how to use the Mercurial Convert extension to split an existing repository.