eigenclass logo
MAIN  Index  Search  Changes  PageRank  Login

Turning Hiki into a simple CMS

I had been looking for something to manage my web presence easily for a long time. I've gone through several phases, corresponding to different tools (always written in Ruby). I recently looked into the software our Japanese overlords are using and evaluated tdiary and Hiki, retaining the latter because a wiki is intrinsically more flexible than a blog, only missing some features I was willing to code.

So I patched Hiki and hacked a couple plugins to turn it into a simple CMS:

  • single-user editing policy
  • changes to make it look less like a wiki to unauthenticated users (i.e. everybody but me)
  • enhanced comment plugin
  • better RSS syndication, based on tags
  • node aggregation to generate blog indices

The most important changes (locking up the wiki and hiding in part its true nature) required but 5 lines of code. The blog-oriented features were easy to implement thanks to Hiki's powerful plugin system. Given these small modifications and additions to Hiki, my modified version keeps all the functionality Hiki had already, and now features:

  • web-based editing by the owner and access through a XML-RPC interface
  • sitemap, table of contents, attachment list generation
  • RSS syndication: recent nodes, RSS feeds restricted to a set of tags
  • blog indices based on tags
  • user comments
  • file uploads
  • CVS/SVN integration for version control of the whole site
  • incremental (AJAXy) search

...

I'm still discovering interesting plugins (pagerank, trackback, referrer tracking, BBS, RSS display, search term highlighting, math support through TeX...). This looks really good.

Here's my patch:


hiki-patch.diff

Sat Oct 22 19:32:42 CEST 2005  Mauricio Fernandez <mfp@acm.org>
  * misc/plugin/aggregate.rb: aggregation plugin, useful for weblogs.
  This plugin allows to aggregate several nodes, blog-style.
Sat Oct 22 19:29:04 CEST 2005  Mauricio Fernandez <mfp@acm.org>
  * Improved RSS support.
  RSS syndication can be restricted to pages matching a set of tags.
  The resulting feed omits whatever comes after the first ^----$ from an entry.
  This can be used to:
  * remove comments
  * create an abstract
Sat Oct 22 19:26:29 CEST 2005  Mauricio Fernandez <mfp@acm.org>
  * Hiki::Plugin#editable?, creatable?: only return true for admin.
  This means that the Create and Edit links won't be shown in the menu if
  there's no write access.
Sat Oct 22 19:24:13 CEST 2005  Mauricio Fernandez <mfp@acm.org>
  * misc/plugin/incremental_search.rb: English adaptation.
Sat Oct 22 19:23:04 CEST 2005  Mauricio Fernandez <mfp@acm.org>
  * misc/plugin/comment.rb: don't update timestamp on comment addition, new format.
Sat Oct 22 19:21:16 CEST 2005  Mauricio Fernandez <mfp@acm.org>
  * Hiki::Command#cmd_edit,cmd_save,cmd_create: Prevent modification by non-admin users.
diff -rN -u old-hiki-0.8.4/hiki/command.rb new-hiki-0.8.4/hiki/command.rb
--- old-hiki-0.8.4/hiki/command.rb	2005-10-26 19:10:57.000000000 +0200
+++ new-hiki-0.8.4/hiki/command.rb	2005-10-22 19:20:44.000000000 +0200
@@ -263,6 +263,8 @@
     end
 
     def cmd_edit( page, text=nil, msg=nil, d_title=nil )
+      raise PermissionError, 'Permission denied' unless @plugin.admin?
+
       page_title = d_title ? d_title.escapeHTML : @plugin.page_name(page)
 
       save_button = @cmd == 'edit' ? '' : nil
@@ -339,6 +341,8 @@
 
     def cmd_save( page, text, md5hex, update_timestamp = true )
       raise PermissionError if @session_id && @session_id != @cgi.params['session_id'][0]
+      raise PermissionError, 'Permission denied' unless @plugin.admin?
+
       subject = ''
       if text.size == 0 && @plugin.admin?
         @db.delete( page )
@@ -412,6 +416,8 @@
     end
 
     def cmd_create( msg = nil )
+      raise PermissionError, 'Permission denied' unless @plugin.admin?
+
       p = @params['key'][0]
       if p
         @p = @aliaswiki.original_name(p).to_euc
diff -rN -u old-hiki-0.8.4/misc/plugin/aggregate.rb new-hiki-0.8.4/misc/plugin/aggregate.rb
--- old-hiki-0.8.4/misc/plugin/aggregate.rb	1970-01-01 01:00:00.000000000 +0100
+++ new-hiki-0.8.4/misc/plugin/aggregate.rb	2005-10-22 19:32:25.000000000 +0200
@@ -0,0 +1,97 @@
+# aggregate.rb
+# Copyright (C) 2005 Mauricio Fernandez <mfp@acm.org>
+# Derived from rss.rb.
+
+def aggregate(tags, page_num = 10)
+  tags = [tags] if String === tags
+  case tags[0]
+  when Array
+    tagsets = tags
+    tagdesc = tags.map{|x| x.join(",")}.join(".")
+  when String
+    tagsets = [tags]
+    tagdesc = tags.join(",")
+  else
+    raise "Either an array of arrays of strings or an array of strings needed."
+  end
+  pages = @db.page_info.sort do |a, b|
+    k1 = a.keys[0]
+    k2 = b.keys[0]
+    b[k2][:last_modified] <=> a[k1][:last_modified]
+  end
+
+  n = 0
+  item_list = ''
+  last_modified = pages[0].values[0][:last_modified]
+
+  output = ""
+  output << %Q!<a href="#{@conf.index_url}?c=rss;tags=#{tagdesc}"><em>RSS feed</em></a>!
+  pages.each do |p|
+    name = p.keys[0]
+    next unless tagsets.any?{|tags| tags.all?{|x| (p[name][:keyword] || []).include? x} }
+    break if (n += 1) > page_num
+    dst = @db.load(name) || ''
+
+    #tokens = @db.load_cache( name )
+    #unless tokens
+    #  parser = @conf.parser::new( @conf )
+    #  tokens = parser.parse( @db.load( name ) )
+    #  @db.save_cache( name, tokens )
+    #end
+    
+    # render without comments
+    full_text = @db.load(name)
+    raw_text = full_text.gsub(%r{^----\n.*}m, "")
+    has_more_text = full_text.size > raw_text.size
+    parser = @conf.parser::new( @conf )
+    tokens = parser.parse(raw_text)
+    tmp = @conf.use_plugin
+    @conf.use_plugin = false
+    formatter = @conf.formatter::new( tokens, @db, Plugin.new( @conf.options, @conf), @conf )
+    content = formatter.to_s
+    #remove {{plugins}}
+    content = content.gsub(%r{<div class="plugin">.*?</div>}m, "")
+    @conf.use_plugin = tmp
+
+    if content and content.empty?
+      content = shorten(dst).strip.gsub(/\n/, "<br>\n")
+    end
+
+    uri = "#{@conf.index_url}?#{name.escape}"
+
+    if true
+      output << <<EOF 
+<div class="day">
+  <h2>
+    <span class="date">#{p[name][:last_modified].utc.strftime('%Y-%m-%d %H:%M UTC')}</span>
+    <span class="title">
+      <a href="#{uri}">#{CGI::escapeHTML(page_name(name))}</a>
+    </span>
+  </h2>
+  <div class="body">
+    <div class="section">
+#{content}
+#{has_more_text ? %[<br/>\n<a href="#{uri}"><em>Read more...</em></a>] : "" }
+    </div>
+  </div>
+</div>
+EOF
+    else
+      output << <<EOF
+<h1>
+  <span class="date">
+  #{p[name][:last_modified].utc.strftime('%Y-%m-%d %H:%M UTC')}
+  </span>
+  <span class="title">
+  <a href=#{uri}>#{CGI::escapeHTML(page_name(name))}</a>
+  </span>
+</h1>
+#{content}
+#{has_more_text ? %[<br/>\n<a href="#{uri}"><em>Read more...</em></a>] : "" }
+<hr/>
+EOF
+    end
+  end
+
+  output
+end
diff -rN -u old-hiki-0.8.4/misc/plugin/comment.rb new-hiki-0.8.4/misc/plugin/comment.rb
--- old-hiki-0.8.4/misc/plugin/comment.rb	2005-10-26 19:10:57.000000000 +0200
+++ new-hiki-0.8.4/misc/plugin/comment.rb	2005-10-22 19:22:57.000000000 +0200
@@ -53,7 +53,10 @@
     if /^\{\{r?comment.*\}\}/ =~ l && flag == false
       if count == comment_no
         content << l if style == 1
-        content << "*#{format_date(Time::now)} #{name} : #{msg}\n"
+        #content << "*#{format_date(Time::now)} #{name} : #{msg}\n"
+        content << "----\n"
+        content << "!!!!#{name} #{format_date(Time::now)}\n"
+        content << "\n#{msg.gsub(/^----$/, "")}\n"
         content << l if style == 0
         flag = true
       else
@@ -65,7 +68,7 @@
     end
   end
   
-  save( @page, content, md5hex ) if flag
+  save( @page, content, md5hex, false ) if flag
 end
 
 def rcomment(cols = 60)
diff -rN -u old-hiki-0.8.4/misc/plugin/incremental_search.rb new-hiki-0.8.4/misc/plugin/incremental_search.rb
--- old-hiki-0.8.4/misc/plugin/incremental_search.rb	2005-10-26 19:10:57.000000000 +0200
+++ new-hiki-0.8.4/misc/plugin/incremental_search.rb	2005-10-26 19:10:57.000000000 +0200
@@ -60,7 +60,7 @@
                               <form method="GET">
                                 #{@conf.msg_search}: <input type="hidden" value="search_orig" name="c">
                                 <input size="50" maxlength="50" name="key" onkeyup="invoke(this.value)" onfocus="invoke(this.value)">
-                                <input type="submit" value="����">
+                                <input type="submit" value="Search">
                               </form>
                               <div id="result">
                               </div>
@@ -75,7 +75,8 @@
                 end
 
                 def search
-                        word = utf8_to_euc( @cgi.params['key'][0] )
+                        #word = utf8_to_euc( @cgi.params['key'][0] )
+                        word = @cgi.params['key'][0]
                         r = ""
                         unless word.empty? then
                                 total, l = @db.search( word )
@@ -90,7 +91,7 @@
                         end
                         header = Hash::new
                         header['type'] = 'text/html'
-                        header['charset'] = 'EUC-JP'
+                        header['charset'] = 'UTF-8'
                         header['Content-Language'] = @conf.lang
                         header['Pragma'] = 'no-cache'
                         header['Cache-Control'] = 'no-cache'
diff -rN -u old-hiki-0.8.4/misc/plugin/rss.rb new-hiki-0.8.4/misc/plugin/rss.rb
--- old-hiki-0.8.4/misc/plugin/rss.rb	2005-10-26 19:10:57.000000000 +0200
+++ new-hiki-0.8.4/misc/plugin/rss.rb	2005-10-22 19:28:46.000000000 +0200
@@ -1,9 +1,33 @@
 # $Id: rss.rb,v 1.22 2005/09/11 10:10:30 fdiary Exp $
 # Copyright (C) 2003-2004 TAKEUCHI Hitoshi <hitoshi@namaraii.com>
 # Copyright (C) 2005 Kazuhiko <kazuhiko@fdiary.net>
+# Modifications Copyright (C) 2005 Mauricio Fernandez <mfp@acm.org>
+
+require 'cgi'
+require 'uri'
+
+# thanks to chris2
+def rss_make_content_relative_to(htmlcontent, root)
+  htmlcontent.gsub(/(<(img|a)\b[^>]*(src|href)=)(["'])(.*?)\4/) do
+    md = $~
+
+    url = URI.parse CGI.unescapeHTML(md[5])
+
+    if url.relative? && url.path !~ /^\//
+      md[1] + md[4] + root + "/" + md[5] + md[4]
+    else
+      md.to_s
+    end
+  end
+end
 
 def rss_body(page_num = 10)
 
+  wanted_tags = [[]]
+  if specified_tags = @cgi.params['tags'][0]
+    wanted_tags = CGI.unescape(specified_tags).split(/\./).map{|x| x.split(/,/)}
+  end
+
   pages = @db.page_info.sort do |a, b|
     k1 = a.keys[0]
     k2 = b.keys[0]
@@ -14,22 +38,29 @@
   item_list = ''
   last_modified = pages[0].values[0][:last_modified]
 
+  if specified_tags
+    tag_desc = specified_tags.gsub(/,/, " AND ").gsub(/\./, " OR ")
+  else
+    tag_desc = ""
+  end
+
   items = <<EOS
 <?xml version="1.0" encoding="#{@conf.charset}" standalone="yes"?>
-<rdf:RDF xmlns="http://purl.org/rss/1.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:content="http://purl.org/rss/1.0/modules/content/" xml:lang="ja-JP">
+<rdf:RDF xmlns="http://purl.org/rss/1.0/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:content="http://purl.org/rss/1.0/modules/content/" xml:lang="en">
   <channel rdf:about="#{@conf.index_url}?c=recent">
-    <title>#{CGI::escapeHTML(@conf.site_name)} : #{label_rss_recent}</title>
+    <title>#{CGI::escapeHTML(@conf.site_name)}#{specified_tags ? ": " + tag_desc : ""}</title>
     <link>#{@conf.index_url}?c=recent</link>
     <description>#{CGI::escapeHTML(@conf.site_name)} #{label_rss_recent}</description>
-    <dc:language>ja</dc:language>
+    <dc:language>en</dc:language>
     <dc:rights>Copyright (C) #{CGI::escapeHTML(@conf.author_name)}</dc:rights>
     <dc:date>#{last_modified.utc.strftime('%Y-%m-%dT%H:%M:%S+00:00')}</dc:date>
     <items>
       <rdf:Seq>
 EOS
   pages.each do |p|
-    break if (n += 1) > page_num
     name = p.keys[0]
+    next unless wanted_tags.any?{|tags| tags.all?{|x| (p[name][:keyword] || []).include? x} }
+    break if (n += 1) > page_num
     src = @db.load_backup(name) || ''
     dst = @db.load(name) || ''
 
@@ -39,16 +70,29 @@
     when 2
       content = word_diff(src, dst).strip.gsub(/\n/, "<br>\n")
     when 3
-      tokens = @db.load_cache( name )
-      unless tokens
-        parser = @conf.parser::new( @conf )
-        tokens = parser.parse( @db.load( name ) )
-        @db.save_cache( name, tokens )
-      end
+      #tokens = @db.load_cache( name )
+      #unless tokens
+      #  parser = @conf.parser::new( @conf )
+      #  tokens = parser.parse( @db.load( name ) )
+      #  @db.save_cache( name, tokens )
+      #end
+      
+      # render without comments
+      full_text = @db.load(name)
+      raw_text = full_text.gsub(%r{^----\n.*}m, "")
+      has_more_text = full_text.size > raw_text.size
+      parser = @conf.parser::new( @conf )
+      tokens = parser.parse(raw_text)
       tmp = @conf.use_plugin
       @conf.use_plugin = false
       formatter = @conf.formatter::new( tokens, @db, Plugin.new( @conf.options, @conf), @conf )
       content = formatter.to_s
+      #remove {{plugins}}
+      content = content.gsub(%r{<div class="plugin">.*?</div>}m, "")
+      if has_more_text
+        uri = "#{@conf.index_url}?#{name.escape}"
+        content << %[<br/>\n<a href="#{uri}"><em>Read more...</em></a>]
+      end
       @conf.use_plugin = tmp
     else
       content = CGI::escapeHTML(unified_diff(src, dst)).strip.gsub(/\n/, "<br>\n").gsub(/ /, '&nbsp;')
@@ -57,7 +101,8 @@
     if content and content.empty?
       content = shorten(dst).strip.gsub(/\n/, "<br>\n")
     end
-    
+
+    content = rss_make_content_relative_to(content, @conf.index_url)
     items << '        '
 
     uri = "#{@conf.index_url}?#{name.escape}"
@@ -111,17 +156,24 @@
   nil # Don't move to the 'FrontPage'
 end
 
+keywords = @options['db'].get_attribute(@options['page'] || 'FrontPage', :keyword)
+tags = keywords.map{|x| CGI.escape(x)}.join(",")
+
 add_body_enter_proc(Proc.new do
-  @conf['rss.mode'] ||= 0
-  if @conf['rss.menu'] == 1
+  if tags.empty?
     add_plugin_command('rss', nil)
   else
-    add_plugin_command('rss', 'RSS')
+    @conf['rss.mode'] ||= 0
+    if @conf['rss.menu'] == 1
+      add_plugin_command('rss', nil, 'tags' => tags)
+    else
+      add_plugin_command('rss', 'RSS', 'tags' => tags)
+    end
   end
 end)
 
 add_header_proc(Proc.new do
-  %Q!  <link rel="alternate" type="application/rss+xml" title="RSS" href="#{@conf.index_url}?c=rss">!
+  %Q!  <link rel="alternate" type="application/rss+xml" title="RSS" href="#{@conf.index_url}?c=rss;tags=#{tags}">!
 end)
 
 def saveconf_rss
diff -rN -u old-hiki-0.8.4/plugin/00default.rb new-hiki-0.8.4/plugin/00default.rb
--- old-hiki-0.8.4/plugin/00default.rb	2005-10-26 19:10:57.000000000 +0200
+++ new-hiki-0.8.4/plugin/00default.rb	2005-10-22 19:25:32.000000000 +0200
@@ -292,15 +292,16 @@
 end
 
 def editable?( page = @page )
-  if page
-    auth? && ((!@db.is_frozen?( page ) && !@conf.options['freeze']) || admin?)
-  else
-    creatable?
-  end
+  admin?
+  #if page
+  #  auth? && ((!@db.is_frozen?( page ) && !@conf.options['freeze']) || admin?)
+  #else
+  #  creatable?
+  #end
 end
 
 def creatable?
-  auth? && (!@conf.options['freeze'] || admin?)
+  admin? #auth? && (!@conf.options['freeze'] || admin?)
 end
 
 export_plugin_methods(:toc, :toc_here, :recent, :br)


how about ... - rg (2005-11-14 (Mon) 15:45:53)

... www.hieraki.org !


mfp 2005-11-14 (Mon) 16:20:11

it seems a very nice piece of sw. and does many of the things I added to hiki, but I'm keeping the latter for now (I prefer the plain file storage and find hiki very hackable).


rg 2005-11-15 (Die) 01:06:50

A plain file db management system in pure Ruby btw is KirbyBase (www.netpromi.com).


mfp 2005-11-15 (Tue) 08:08:29

Is the activerecord adapter ready? Hieraki also uses some SQL which would need to be changed. But even if the DB is stored in plain text files, hieraki seems to be using a fairly contrived schema (I see it uses a B-tree)... definitely too much work for my lazy self.


jo 2005-12-28 (Mit) 03:18:05

A very 'hackable' Unix-based blogging system is nanoblogger.sf.net!


pete 2006-01-03 (Tue) 10:52:30

i too went around kicking a lot of tires for basic wiki functionality that i could share with my friends. Hiki looks good to me too. Can you update how your experience has been with hiki? Any hacker vulnerabilities? I'd like to tweak hiki to use RedCloth, too....


mfp 2006-01-03 (Tue) 16:42:49

jo: seems very powerful... maybe it'd benefit from a rewrite in Ruby ;) (cat+grep+sed+bash sounds like pain) Plain Ruby (and only Ruby) might be more portable than bash and friends.

pete: So far I'm fairly satisfied with hiki. I found rather easy to add fairly unusual things like syntax coloring for Ruby source and math support (hiki ships with LaTeX-based support but I developed a derived version built atop troff since the former wasn't installed in the server). I haven't had any security problems so far. There were two security advisories last year (before I began using hiki); as far as I can tell, they handle this seriously. As for the cons, I'd note:

  • the comment system built on top of the wiki might not be powerful enough (it implies some limitations regarding formatting etc.)
  • although there's an XML-RPC interface, I haven't used it because some plugins like attachments/comments don't expose any such interface AFAIK. But I haven't really looked into this yet (I was too concerned about security and disabled XMP-RPC right away, not having found the time to read the documentation and enable it properly since) so I might be wrong.
  • even if you configure it properly to plug into a svn/cvs repository (which I haven't done yet) to put the wiki under version control, attachments will be managed independently. But to be fair, how many blog engines out there support full content versioning?

As for using different markup languages: having read the source code, I think it should be fairly easy. Actually, Hiki ships with the default markup (hikidoc), an enriched version with math support ("math"), and RD markup. I've thought of implementing per-node markup selection, but haven't managed to write it yet.

yet another plugin - Jo (2005-11-14 (Mon) 09:34:17)

Yet another potential plugin you might want to check out is deplate.sf.net!


mfp 2005-11-14 (Mon) 16:08:29

looks good --- some things would be hard to map (e.g. contents generated by the plugins that output HTML), but basic conversion to LaTeX and some docbook subset should be doable. I'm adding this to my "would be nice to do" list.

patch: stay updated - maurizio (2005-11-05 (Sat) 23:56:42)

hello. thank you for posting this. i'm a ruby newcomer and i've evaluated a host of php-based cms (i'm sticking with drupal right now), but partially unsatisfied. drupal is mostly community-oriented and too rigid in my opinion (posed to becoming a bloatware ...). beginnining to look at ruby and rail, i discovered your site and i'd like to give a try to hikiwiki and your pacth. i'll stay tuned via rss feeds. let me know if i can be of help. thank you.

Patch: How To? - Timothy Fisher (2005-11-01 (Tue) 11:26:20)

Can you tell me how I'd apply this patch to a standard Hiki installation?

Thanks, Tim


mfp 2005-11-01 (Tue) 12:13:52

patch -p1 < hiki-patch.diff run under the hiki-0.8.4/ dir should do. I'm soon uploading a much improved patch with support for RSS 2.0, aggregate & feed caching, better incremental searching, enhanced mail change notification...


Last modified:2005/11/01 03:14:33
Keyword(s):[ruby] [hiki] [cms]
References:[Ruby]