UBB.Sitemaps · Wiki

What Is Covered

• Summary
• About the Modification
• Additional Files
     • Supplied robots.txt Instructions
• Caching
• Demonstration
• Install Instructions
• Usage Instructions
• Settings
     • Configuration Settings
     • Database Connection Information
     • Cache & Optimization Options
• Human Readable Permalinks
• Human Readable XML Styling
• Submissions
• UBB.threads Version & URLs
• Changelog
• Troubleshooting
     • Invalid Date Error
• Frequently Asked Questions
     • What does purchasing a license entitle me to?
     • What are the security aspects?
     • What is a Sitemap?
• Useful Links
• Thank You


 

Summary

This page represents the full documentation for the UBB.Sitemaps script by VNC Web Services.


 

About the Modification

• Requirements: UBB.threads 7.x (not compatible with UBB.threads 6.x or below)
• Current Version: 20210901
• License Information: This script is not freeware, it requires a license; by purchasing a license you're granted usage on 1 (one) UBB.threads installation. Any additional installations require an additional license. Updates are available for 1 year after initial purchase; after this timeframe, to aid in development costs, you must renew your access to updates (see the purchase link below for further information). You cannot freely provide this script to others without written permission from the author.
• Pricing: $80 (Purchase Here)
• Support: Support is available on the UBBDev thread [7.x] UBB.Sitemaps.

UBB.Sitemaps is a script that we created due to some reported search engine "discovery" issues with the way that URLs where initially written in the UBB.threads product that some users where reporting. Not to mention that a lot of users had moved from previous or competing products and wanted to see their URLs immediately discovered in the Google, Bing, Yahoo, or other search
engine(s).

This script aids in the "discovery" portion of search engine crawling; instead of waiting and hoping that your URL's get crawled, an XML Sitemap alerts Google (or any crawlers which allow sitemaps) as to what URLs exist on your forums, so they know what to crawl based on what you tell them.

Most search engines will take your feed once a week or so and run it through to determine new URLs, on a Google Webmaster Tools account, you can browse exactly how many URLs have been included, as well as when the last time they have hit your site.

Please note however, that Google will still crawl your site and try to perform its own discovery, so please do not use a robots.txt entry to block it (as it must be able to crawl the content provided in the sitemap in order to populate your entries).

The configuration options allow you to specify which type of sitemap you wish to use and whether you wish to plugin threaded or flat view. The sitemap type and view type can both be set either by hard coding the appropriate variable or via a URL string.


 

File Listing

Your install archive will contain several files, namely:
• cache/sitemap/ - A blank directory to show the proper nesting for the default cache directory.
• css/sitemap.css - A CSS Stylesheet for XML Feeds.
• readme.txt - A brief readme which gives basic tips on getting started, and a link here to the full documentation.
• robots.txt - A listing of instructions for crawlers which support robots.txt; see below for a listing of the supplied instructions.
• sitemap.xsl - An XSL Style.
• ubb.sitemap.php - The UBB.Sitemaps script itself.

 

Supplied robots.txt Instructions

Though not required, we recommend applying the items in the robots.txt file in the attached folder so that Google and other search engines do not attempt to crawl areas in which they commonly get trapped. Please be sure that if you utilize the items in the robots.txt file that you update the path to reflect that of your forums.




 

Caching

In v0.6 we added several caching and optimization areas to the script; these will help alleviate load issues from multiple clients grabbing your map. Please note that at times there may be some strict server configurations which won't work well with these options, your mileage may vary.

The cache option will build a copy of the output and store it for a defined period of time, any user accessing the file during this period will be served the cached file vs the live file (with the exception of the initial visiting user who will be served the live file.

The exception header option will allow you to set a date "in the future" when the content of the file "should" change, we default this to 24 hours so the client should cache this file on their side for this time frame.

To utilize caching you will need a cache/sitemap/ folder (from the script instance) that is chmodded 777.

The SHA1 hash of the test URL (for the cache directory) is e0f94273f1b777a695d0f53fd7acc4c111942675.


 

Demonstration

The UBB.Sitemaps script offers several sitemap generation styles, these are:
• UBB.Sitemaps - ASP Sitemap
• UBB.Sitemaps - Link Sitemap
• UBB.Sitemaps - RSS Sitemap
• UBB.Sitemaps - Sitemap Index
• UBB.Sitemaps - Text Sitemap
• UBB.Sitemaps - XML Sitemap

In v0.9 we introduced the ability to style your XML and Test output pages; which make them more human readable (and functional in the case of the XSL style):
• UBB.Sitemaps - Style (CSS)
• UBB.Sitemaps - Style (XSL)


 

Install Instructions

Open the ubb.sitemap.php file in a plain text editor (such as notepad or EditPlus) and adjust the variables for your environment; all of the settings include comments which give information on the setting as well as the defaults.



After customizing the script settings you'll need to upload the script to your web root; in order for Google or other search engines to read that the sitemap belongs to your domain, the sitemap generator must be in the html document root (https://www.example.com/).


 

Usage Instructions

We've built the sitemap generator to be quite flexible, there are several options and parameters that you can customize for your live feed of your sitemap for numerous uses scenarios as:
• Sitemap Types: 0/Test, 1/XML, 2/Text, 3/html, 4/RSS, 5/ASP, 6/SitemapIndex (Default: 6/SitemapIndex)
• Use SE Friendly or Search Engine Optimized URLs (Default: 1/Yes)
• Use ShowFlat or ShowThreaded. (Default: 0/ShowFlat)
• Offsetting links to break up large forum sitemaps into manageable chunks. (Valid values are from 0-50000; Default: 10000)

And all variables can output either "as default" by directly accessing ubb.sitemap.php or by defining a parameter in the URL, such as:
Google would be happiest with: https://www.example.com/ubb.sitemap.php?type=6&se=1 or https://www.example.com/ubb.sitemap.php?type=1&se=1 (That's type=6 for a Sitemap Index (or type=1 for XML), and se=1 for SEO/Spider Friendly URL's. We recommend that everyone just default to using a SitemapIndex as it future-proof's your site (as a Sitemap can only have up to 50k links; however, loading the ceiling maximum of data, to be loaded at once, can cause strain on your server. Using a SitemapIndex will break your Sitemap up into more manageable chunks).

In the past, yahoo was more comfortable with RSS feeds, but recently they've also adapted to work well with Google's XML sitemaps.

As this script pulls data live from the database, it is unwise to link directly to the various output options unless it's being fed into search engines as there is a potential for a lot data output. We recommend always using the caching option.

To test to ensure it is working before providing the feed URL's to search engines, please load your sitemap generator with the following parameters (as the preview uses a small limit it shouldn't take longer than a couple of seconds to execute):
https://www.example.com/ubb.sitemap.php?type=0&se=1

The sitemap index only works with XML sitemap feeds based off of the sitemaps.org standard; as these are only valid under XML sitemaps they display the output according to that standard only. To grab the sitemap index (should you have the need to utilize the offset option) please access it as:
https://www.example.com/ubb.sitemap.php?type=6&se=1

If you are _NOT_ going to use a SitemapIndex and want to load the entire dataset at once, you'll want to change the "offsets" option to not utilize them, otherwise it'll load the amount of data up to the default offset chunk size (of 10k).


 

Settings

 

Configuration Settings

     $conf["connection"] = 1; // 0/Set DB Info (below); 1*/Pull data from UBB.threads Config.
     $conf["domain"] = ""; // The domain that your forum is installed on (Ex: example.com); if blank will read the "_SERVER["HTTP_HOST"]" PHP variable (recommended).
     $conf["folder"] = "forum"; // The folder that your UBB.threads is located in, Example: https://www.example.com/forums/ would be "forums/" (without trailing slash)
     $conf["extension"] = 1; // Use HTML extension? 0/No, 1/Yes*
     $conf["frequency"] = 4; // 0/Never, 1/Always, 2/Hourly, 3/Daily, 4*/Weekly, 5/Monthly, 6/Yearly, 9/No Display
     $conf["friendly"] = 1; // Use SE Friendly/SEO Links? 0/No, 1*/Yes
     $conf["lastmod"] = 0; // Display last modified time? 0*/No, 1/Yes
     $conf["offsets"] = 1; // 0/No, 1*/Yes if SitemapIndex (See "Type" Above)
     $conf["pages"] = 1; // Build links for UBB.threads Pages 0*/No, 1/Yes
     $conf["pagesrw"] = 1; // Respect Apache Rewrite Rules for UBB.threads Pages? 0*/No, 1/Yes
     $conf["rewrite"] = 0; // Respect Apache Rewrite Rules? 0*/No, 1/Yes
     $conf["ssl"] = 0; // 0*/Native (based on url accessed), 1/Always Show SSL URLs
     $conf["style"]["xml"] = 0; // 0*/No, 1/CSS (sitemap.css), 2/XSL (sitemap.xsl)
     $conf["style"]["name"] = "sitemap.xsl"; // The name of the sitemap style; an example css/sitemap.css and sitemap.xsl are supplied.
     $conf["timezone"] = "America/Los_Angeles"; // Your localized timezone; default: America/Los_Angeles. See: https://php.net/manual/en/timezones.php
     $conf["title"] = "Community Name"; // You can set this to change the name of your feed. By leaving this by default (or blank) and selecting to use data from your config (below) it will use your community name.
     $conf["type"] = 6; // What type to use by default? (See Above: 1/2/3/4/5/6*/7/8)
     $conf["version"] = 3; // UBB.threads Version, 0/v7.0 - v7.2.2, 1/v7.3 - v7.4.2, 2/v7.5 Series, 3*/v7.6 Series
     $conf["view"] = 0; // What view would you like to use? (See Above: 0*/1)

 

Database Connection Information

     $conf["db"]["password"] = "sql_password"; // This should be the password to your database.
     $conf["db"]["prefix"] = "ubbt_"; // This should be the table prefix of your UBB.threads installation.
     $conf["db"]["name"] = "sql_database"; // This should be your database which holds your UBB.threads installation.
     $conf["db"]["server"] = "localhost"; // This should be the hostname that you use to connect to MySQL.
     $conf["db"]["username"] = "sql_username"; // This should be the username to your database.



 

Cache & Optimization Options

     $conf["expires"]["status"] = 1; // Should we set an expires header? = 0 = No, 1 = Yes (Default: 1)
     $conf["expires"]["time"] = 86400; // In Seconds (Default: 86400)
     $conf["cache"]["directory"] = "cache/sitemap/"; // The directory you intend to use for the storage of any cache files; requires $conf["cache"]["status"] to be 1. Example: "cache/sitemap/"; directory MUST be chmodded 777.
     $conf["cache"]["status"] = 0; // Should we use the cache? (Requires the "cache_dir" value to be populated.) 0 = No, 1 = Yes (Default: 0)
     $conf["cache"]["time"] = 86400; // In Seconds, (Default: 86400)
     $conf["gzip"] = 1; // Use GZip Compression for serving pages (can add some CPU load). 0 = No, 1 = Yes (Default: 1)




 

Human Readable Permalinks

In v1.0 we added the ability to specify human readable permalinks. These links are programmed into both sitemap scripts (UBB.Sitemaps and UBB.Membermaps) and function via Apache Rewrite Rules. These rules go into your .htaccess file (if supported by your web server) and will redirect the following urls:
https://example.com/sitemap_index.xml -> Sitemap Index page -> ubb.sitemap.php?type=6&se=1
https://example.com/sitemap_listing.xml -> Forum Listing page -> ubb.sitemap.php?type=7&se=1
https://example.com/sitemap_members.xml -> UBB.Sitemap Member Listing -> ubb.membermap.php?type=6&se=1
https://example.com/sitemap_testing.xml -> UBB.Sitemaps Test Page -> ubb.sitemap.php?type=0&se=1
https://example.com/ubb_membermap/### -> Offset Pages
https://example.com/ubb_membermap/###.html -> Offset Pages
https://example.com/ubb_sitemap/### -> Offset Pages
https://example.com/ubb_sitemap/###.html -> Offset Pages




 

Human Readable XML Styling

In v0.9 we added the ability to style the output of XML feeds so that a human can better read and interact with your sitemap. This is controlled with the $conf["style"]["xml"] and $conf["style"]["name"] variables within the Configuration Settings.

You can opt to use a simple CSS Stylesheet or a XSL Style which was supplied by id242; the XSL file is what we recommend using as it makes the XML output interactive. A demo of this can be found in the Demonstration section above.


 

Submissions

Not all search engines will check back automatically to re-check your sitemap; in fact, we've only noticed that Google will re-check on its own. You'll need to check back with other search engines to re-submit your sitemap if they do not auto-check it for you.

We recommend providing the following URL to search engines that support XML sitemaps:
https://www.example.com/ubb.sitemap.php?type=1&se=1&offsets=0

Those switches, as mentioned in the Usage Instructions will generate an XML sitemap with Search Engine Friendly/SEO URLs while opting not to use Offsets.


 

UBB.threads Version & URLs

URLs (and associated variables) have changed several times in the coarse of UBB.threads, below is some brief information for the $conf["version"] variable:
0 = v7.0 - v7.2.2 - Original style, non Search Engine friendly, URLs.
1 = v7.3 - v7.4.2 - Global Moderators where introduced in 7.3, which changed the guest user to user #5.
2 = v7.5 Series - v7.5 introduces SE Friendly URLs, which provides the topic title within the link itself.
3* = v7.6 Series - v7.6.0 introduces Search Engine Optimized (SEO) URLs, which provides the topic title within the link itself, and optimized the display for.




 

Changelog

 20191222 (December 22nd, 2019)
• Added: Debug mode (ubb.sitemap.php?debug=1) will allow you to see your UBB.Sitemap settings and a forum overview in an easy to read page.
• Added: A new version check will display a version notice on the debug page.
• Added: A switch has been added for the beta group which will display if you're running a beta release or not. This will allow the new version check to be able to display notices for final builds.
• Added: The UBB.threads Pages display respects the private mode status within UBB.threads Pages.
• Added: Introduced basic mobile compatibility on HTML generated pages (type=3)
• Fix: Fixed a bug in UBB.threads Pages detection.

 20190818 (August 18th, 2019)
• Added: Support for UBB.threads Pages
• Changed: HTML Sitemaps no longer build in a table, instead they will build as a listing of links with a break
• Changed: Switching from Version Numbers to Build Dates (20190818)
• Changed: The HTML extension can now be disabled, and will respect the UBB.threads configuration value
• Fix: Code Cleanup and Reformatting
• Fix: The Forum Listing will check that the forum is enabled before displaying it

 v1.0/20180828 (August 28th, 2018)
• No longer in Alpha as the script has been tested extensively on many server configurations.
• Added a listing of all forum names (type 7), which is linked from the Sitemap Index page.
• Added an option to not display the frequency line.
• Added an option to not display the lastmod line.
• Added an option to not display the .html extension for sitemap files.
• Added support to always produce HTTPS (SSL) URLs (useful for sites on a CDN where the local server may still try to produce non-HTTPS links when using wildcard HTTPS urls).
• Added support for including the UBB.threads configuration for sites running UBB.threads from the webroot (not in a folder).
• Added support for specifying a domain (useful if your UBB.threads install is on a subdomain and you wish to have UBB.Sitemaps located on your main domain).
• Added support for human readable permalinks with Apache Rewrite Rules
• Removed the need for a trailing slash in the folder config.
• Changed the MySQL database connection to utilize MySQLi for PHP7 support.
• Minor code changes for script performance.

 v0.10 (October 26th, 2015)
• All queries have been cleaned up, the tally that determines a SitemapIndex's pages is done entirely server side as a single database query. Additionally the query options where combined and rewritten so that one query loads all of the needed data for the building of each Sitemap.
• Several configuration options where changed in both v0.9 and v0.10, it is recommended that you compare the options at the top of your page from your current build to the newest version that you wish to upgrade to so that you can see the changes in options and defaults.
• We now recommend everyone run as a SitemapIndex with offsets enabled as it makes your site future-proof to the amount of data you have in your Sitemap as well as break the data up into manageable chunks that should negate some server strain on large forums.
• Added support for date_default_timezone_set; you can now set the timezone for the generation times supplied by your sitemap (defaults to Pacific Standard Time for those who do not set an option).
• Support for virtual SSL over a CDN using "HTTP_X_FORWARDED_PROTO".
• Fixed a bug with caching.
• Fixed a time issue with the formatting previously used.
• Removed the recommendation of blocking user profiles in the robots.txt file as there could be some potentially valuable information as far as data is concerned.

 v0.9 (April 2nd, 2015)
• All configuration items are now stored in a $conf array.
• ASP sitemaps no longer populate both the title and description tags as they where duplicated and thus added to the bulk of sitemap pages.
• General script code cleanup, reformatting, and restructuring.
• The test string is now type 0; previously, type 0 was not utilized and would cause errors if indicated. Type 7 no longer works for any type of generation.
• We're now utilizing more current sitemap standards; previously we where utilizing earlier versions.
• There is now support for v7.6.0 style SEO URLs
• The site name parameter is no longer passed to the thread title as this just added a lot of extra bulk to the pages.
• The last reply time of a topic is now utilized in XML sitemaps.
• Utilization of an XML Stylesheet is now supported for XML Feeds; see the supplied css/sitemap.css and sitemap.xsl files for basic options.
• For compatibility for users hosting their UBB.threads install in the web root, the $conf["folder"] variable now requires a trailing slash.
• RSS Sitemaps now utilize the <ttl> tag which is fed from the $conf["cache"]["time"] variable.

 v0.8 (March 28th, 2013)
• Code Cleanup (no more Undefined variables)
• More efficient handling of SSL detection
• Preview no longer allows cache (as it should always be "live" for testing)

 v0.7 (May 26th, 2011)
• Search Engine Friendly URLs now properly match the UBB.threads' Search Engine Friendly URLs (including a more powerful SEO score for URL structure).
• You can now set the frequency that you want Search Engines to check back for new content using the $use_frequency tick.

 v0.6 (August 20th, 2009)
• This is known as the Caching and Optimization update since improvements where added to increase the speed of your install.
• Added a simple time-based caching system which will allow you to set how long the output is to be cached; this cache is built on the first visit to the script of each URL type (if a value is changed in the requesting URL it'll result in a separate caching of that output of data). The default caching time is 24 hours (this value is entered in seconds).
• Added an expires header to the code to allow client browsers (or search engines (if they support it) to cache the output of the sitemap script to their local machine for a defined period of time (default 24 hours, entered in seconds).
• Added the ability to serve the output of the script via GZip to aid in download times (basically serving a compressed dataset to the user). GZip has been known to cause some minor CPU load, but most virtual host providers should be more than capable of serving content.
• Fixed numerous cosmetic display issues and fixed a bug that's been in since ~v0.2 which made it so that the "blank" display didn't parse any data (vs falling back to the defaults you display).



 v0.5 (February 15, 2009)
• Removed "Priority" tag as it was optional and providing duplicate errors from Google.
• Removed some "test code" from the header of the script left in from 0.4.
• Fixed a bugglet with the "read from config" option which was added in 0.4.
• Added a "tick" to config which allows you to display the optional "changefreq" tag.
• Added the ability to pull the community name from your UBB.threads config as the feed name.
• Updated the changelog format to fix spelling and grammatical errors introduced in 0.3.

 v0.4 (May 1st, 2008)
• Version 0.4 introduces a "preview" URL set for testing connections. This preview set can be accessed as: https://www.example.com/ubb.sitemap.php?type=7&se=1
• The db connection data no longer has to be hand written into the UBB.Sitemap script; so long as you set the path to your UBB.threads Configuration directory and have "connection" set to 1, it will read the connection data from your UBB.threads configuration file.
• Additionally the Preview URL set in the "usage" area of the readme has been updated to reflect the 0.3a update. Also included in this update is compatibility with the permission update for 7.3.

 v0.3 (June 17, 2007)
• Support for SitemapIndex's have been built in; this works well with the "offset" feature added in v0.2. The sitemap index will look at the xml sitemap data and build the proper number of "valid" links to offset pages. This is extremely useful for larger forums as they can set what level they want to offset their data (as to not go over the 50k maximum link limit with most providers) and they just need to submit one URL (versus numerous to cover additional sitemap pages.

 v0.2 (March 20th, 2007)
• This version introduces offsetting; Google has a maximum of 50k links allowed in a sitemap, some larger (huge) forums have more than 50k threads on their forums. This setting is enabled by default, and can be utilized by using the offset variable in your URLs submitted to Google/Yahoo. Additionally, it can be disabled completely in the configuration file.

 v0.1 (March 1st, 2007)
• Initial version released to the public.


 

Troubleshooting

 Invalid Date Error (1969)
This error is caused from a malformed row (or rows) in the MySQL database (either from a failed prune, topic creation, etc) which generally has blank data inserted for the row. As a timestamp of "0" in the database cannot be translated to a valid date (UBB.threads, like most other scripts stores dates in the database in the Unix Epoch format) it will be translated as a date prior to the Unix Epoch (in this case, 1969).



You'll need to edit the _POSTS and _TOPICS tables, you'll want to sort the database by the timestamp field (post_created for the posts table, topic_created for the topics table; starting with the topics table) and look for any rows returning a "0"; you'll then want to delete the rows that return a blank set of data (generally no subject or post data) and remove those rows from the database.

After purging the rows from the database you'll want to make sure that you clear the sitemap cache (by default it is not enabled, and the default location would be "cache/sitemap/" (delete any .html files in this folder)).


 

Frequently Asked Questions

•  

What does purchasing a license entitle me to?

By purchasing a license to the UBB.Sitemap script you're entitled to lifetime support of the script for the lifetime period of the script. You also receive access to a year of updates for free. After the year period; updates are purchased in blocks of 1 year for $80/yr to cover development costs. Please note that you are entitled to install this script on ONE domain per license; additional domains will require additional licenses.


•  

What are the security aspects?

The UBB.Sitemap script will ignore all threads that are posted in private forums (basically, if a user who is logged out cannot *see* a thread in a forum then the script will not publish links in that forum). Also, threads that are not approved will not show either (as they're not visible to guest users).

•  

What is a Sitemap?

A sitemap will take and provide all of your current threads to search engines, thus eliminating the wait of "new discovery" of new pages on your forums.

You can read about the benefits of using sitemaps via the below quoted article from Google Webmaster Tools.






 

Useful Links

UBBDev discussion for UBB.Sitemaps
Script Information on VNC Web Services
Bing Webmaster Tools
Google Search Console


 

Thank You

I'd like to take the time to fully acknowledge id242 for assisting me with several tasks for this script, and for renewing my interests in everything when I was at my wit's end. Thanks to your support, hard work, and friendship, everything is looking up :).



Posted By Gremelin Posted on October 2nd, 2014 · Updated on December 6th, 2023

Return to the Wiki Index
Return to the Category "Modifications"
Submit a Wiki for Review