Google sitemap agent and update it to Google with a ’ping’ service

Google has a cool little website called ‘Google Webmaster tools’.
One of the features of Webmaster tools is the possibility to upload the sitemap of your site. This will improve the indexing of your website. A Google sitemap is a XML file which has the following structure:

<?xml version=”1.0″ encoding=”UTF-8″ ?>
<urlset xmlns=”http://www.google.com/schemas/sitemap/0.84“
xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance“
xsi:schemaLocation=”http://www.google.com/schemas/sitemap/0.84
http://www.google.com/schemas/sitemap/0.84/sitemap.xsd“>
<url>
  <loc>http://www.domain.com/dir/page.html</loc>
  <lastmod>2010-10-27</lastmod>
  <priority>1</priority>
  </url>
</urlset>

Dynamic sitemap
We want to update this sitemap automatically with a Lotusscript agent when we create or modify a page. We have created a view called ‘vw-web-Sitemap’ which has the structure of our content in it in hierachical format. It loops through the view, get the field ‘urlkey_seo’ and the last modified date and places them in the XML. The field ‘urlkey_seo’ contains the actual path of the page (ex: /products/coolproduct1/specs).

We created a small function to get the date in the correct format (yy-mm-dd). In our case we use three templates (forms), and we give a different priority to it, the homepage should be 1 (high), our products should have a priority of 0.8, and all other pages should have a priority of 0.5 (medium).

The agent has the following code:

%REM
Agent web-GoogleSitemap
Created Oct 27, 2010 by Ferdi Verlaan/Aas
Description: Creates a google sitemap from the content of the sitemap view
%END REM
Option Public
Option Declare

Sub Initialize

On Error GoTo errh

Dim session As New NotesSession
Dim db As NotesDatabase
Dim view As NotesView
Dim doc As NotesDocument
Dim rel As String        

Set db = session.Currentdatabase
Set view = db.Getview(“vw-web-Sitemap”)
Set doc = view.Getfirstdocument()

‘header information for XML        
Print {Content-Type: application/xml”}
Print {<?xml version=”1.0″ encoding=”UTF-8″?>}
Print {<urlset xmlns=”http://www.google.com/schemas/sitemap/0.84″
        xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance”
        xsi:schemaLocation=”http://www.google.com/schemas/sitemap/0.84
        http://www.google.com/schemas/sitemap/0.84/sitemap.xsd”>}

Do Until doc Is Nothing

If doc.Form(0) = “cnt-Diensten” Then
rel = “0.8”
elseIf doc.Form(0) = “cnt-Start” Then
rel = “1”
Else
rel = “0.5”
End If

Print {<url>}
Print {<loc>http://www.aashq.nl} & doc.urlkey_seo(0) & {</loc>}
Print {<lastmod>} & genDate(doc) & {</lastmod>}
Print {<priority>} & CStr(rel) &{</priority>}
Print {</url>}

Set doc = view.Getnextdocument(doc)
Loop        

Print {</urlset>}

Exit Sub

errh:
Print “Error on line “ & Erl & ” – “ & Err & ” – “ & Error & ” in “& GetThreadInfo(1)        
Exit Sub

End Sub

%REM
Function lenDate
Description: checks the length of the day or month value
%END REM
Function lenDate(ndt As String) As string

If Len(ndt) = 1 Then
lenDate = “0” + ndt
Else
lenDate = ndt
End If

End Function
%REM
Sub genDate
Description: Creates a yy-mm-dd date string
%END REM
Function genDate(doc As NotesDocument) As string

Dim ndt As New NotesDateTime(CStr(doc.LastModified))
Dim y As String
Dim m As String
Dim d As string

y = CStr(Year(CStr(ndt.Lslocaltime)))
m = lenDate(CStr(Month(CStr(ndt.Lslocaltime))))
d = lenDate(CStr(Day(CStr(ndt.Lslocaltime))))

genDate = y + “-“ + m + “-“ + d

Exit Function

End Function

This agent will create a nice XML output for Google Sitemaps. You can submit it to Google with Webmaster tools > Siteconfiguration > Sitemaps.

Ping sitemap
After submission Google will index the sitemap periodically, but wouldn’t it be great to let Google now that the sitemap has been updated? That is possible through a special PING website of Google. The url of this ping service is: http://www.google.com/webmasters/sitemaps/ping? and then the url of your sitemap.

We have created a ping agent for Lotus Notes which automatically runs after a modification or creating of a new document.
The agent code is listed below (works only from Windows):

%REM
Agent web-pingSitemap
Created Nov 11, 2010 by Ferdi Verlaan/Aas
Description: Pings the sitemap to Google services
%END REM
Option Public
Option Declare

Sub Initialize
Dim result As String
result = pingSitemap(“http://www.google.com/webmasters/sitemaps/ping?sitemap=http://www.domain.com/website.nsf/web-GoogleSitemap.xml”)
End Sub

Function pingSitemap(strUrl As String) As String
 
Dim oHTTP As Variant
Dim strReturn As String

Set oHTTP = CreateObject(“Microsoft.XMLHTTP”)

oHTTP.open UCase$(“POST”), strUrl, False, “”, “”
oHTTP.setRequestHeader “Content-type”, “application/x-www-form-urlencoded”
oHTTP.send(“”)

pingSitemap = oHTTP.responseText
Set oHTTP = Nothing

End Function

Good luck with implementing this on your own website.