Creating a simple XMLish parser

[ssjwizard]

Ok well as some of you may know I have been building a Java + LWJGL based game engine over the past year, and I have finally wrapped up my 3rd stage prototype game. So going forward now that I have an established set of objects, entities, and so fourth Its time to start making tools that can do bits of the game building for me.

Before anyone brings it up I am fully aware that there are indeed existing Java XML parsing libraries, however, I will not be using one for a few reasons. One of which is distribution size, another is potential platform related issues, along with a bit of my own stubbornness, and not caring to read any of there license agreements. So it falls upon me to create a system of parsing resources from text.

My plan is to attack this with a 2 part setup. Part 1 is the ResourceParser class and Part 2 will be the ResBlock class. Now this is just the initial idea and it really needs to be worked out a bit before I get neck deep down a code path thats going to bite me in the ***.

So my thoughts on doing this is that the ResourceParser is responsible for opening the text file and wrapping up the text into String arrays. It would then push that array into a new ResBlock object.

ResBlock is then responsible for handling the many lines of text. I am thinking a field known as Identity which will contain the opening tag of the given array. For example a block structured like

[resource]
.
.
.
[/resource]

Its Identity would be "resource". I am thinking It will need a few methods as well. Probably something like containsSubBlock which will search the String array for any potential sub blocks such as:

[imageResource]
file-id = ...
resource-id = ...
etc..
[/imageResource]

if a sub block is found it would return true. This would then allow the ResourceLoader to ask for the blocks. The thought is that a [resource] block could hold multiple sub resources. It would also need to be able to return these blocks one at a Time so that ResourceLoader can then take each individual resource and build them.

If anyone has experience parsing XML, or building a similar parsing tool, or even just working with one I'd absolutely ecstatic to hear your input on this. Im open to suggestions on approaching this idea from a totally different angle if it seems more appropriate. So let me know what you think people!

 

[ssjwizard]

Ok well I decided that I would post up a bit of what I have hammered out on this so far. I am still missing the ResourceLoader which will convert the ResBlock data into game objects, and put them into ResourceManager so game elements can access them.

So the basic idea behind the design here is simple. I can create resources which contain any type of data and index them quickly at game initialization. The resources will be defined in plain text blocks which I am building GUI tools to generate. So I will quickly be able to build a master image resource and chop it up into many smaller resources with unique identifiers.

The ResourceParser is responsible for reading the text files and creating an array of ResBlocks. The ResBlock recursively finds any inner blocks at time of instantiation. The array will then pass over to ResLoader and depending on what the "identity" of the block is it will either find sub blocks or create resources to be passed into the manager.

There are many advantages to making every resource abailable in a single location. For me the next game were producing will include online multiplayer functionality. The command messages will be relying on the ResManager to be capable of allocating the exact same resources on different systems. This also prevents the need to serialize any data across the network connection.

package midnight.util;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.ArrayList;

import org.newdawn.slick.SlickException;


public class ResourceParser {
  
  private ArrayList<String> input;
  private ArrayList<ResBlock> blocks;
  
  public ResourceParser(){
    this.input = new ArrayList<String>();
    this.blocks = new ArrayList<ResBlock>();
  }//end constructor
  
  public ArrayList<ResBlock> parseResourceFile(String fileName) throws Exception   {
    
      input.clear();//clear the buffer
      FileReader file;//file reference
      file = new FileReader(fileName);
      BufferedReader read = new BufferedReader(file);
      
      boolean check = false;
      String test;
      while(!check){
        
        test = read.readLine();
        if(test == null){
          check = true;
        }
        else{
          this.input.add(test);
        }
        
      }//end of reading lines

      this.blocks.clear();//clear existing blocks

      int initial = 0;
      String begin = "[";
      String close = "[/";
      String end = "]";
      String id;
      for(int i=0; i<this.input.size(); i++){

        if(input.get(i).contains(begin) && input.get(i).contains(begin)){
          
          initial = i;
          id = input.get(i);
          int fist = id.indexOf(begin)+1;
          int last = id.indexOf(end)-1;
          id = id.substring(fist, last);
          boolean endFound = false;
          
          for(;i<input.size();i++){

            if(input.get(i).contains(close + id + end)){

              String[] data = (String[]) this.input.subList(initial, i).toArray();
              ResBlock temp = new ResBlock(data,id);
              this.blocks.add(temp);
              initial = i;
              endFound = true;
            }//find closing tag
            
          }//end loop
          
          if(!endFound){
            throw new SlickException("Error parsing resource, end tag not found in: " + 
                fileName + " section : " + id);
          }//check if resource is complete

        }//end if id found

      }//end parsing loop
      
      
      return this.blocks;
      
    
  }//end parse resource file
  
}//end resource parser

package midnight.util;

import java.util.ArrayList;

import org.newdawn.slick.SlickException;

public class ResBlock {
  
  private final String identity;
  private ArrayList<String> block;
  boolean innerBlocks = false;
  private ArrayList<ResBlock> subBlocks;
  private int last =-1;
  
  public ResBlock(String[] resource, String type) throws SlickException{
    
    this.identity = type;
    this.block = new ArrayList<String>();
    this.subBlocks = new ArrayList<ResBlock>();    
    
    for(String ln : resource){
      this.block.add(ln);
    }//add all lines
    
    int initial = 0;
    String begin = "[";
    String close = "[/";
    String end = "]";
    String id;
    
    for(int i=1; i<this.block.size()-1; i++){

      if(block.get(i).contains(begin) && block.get(i).contains(begin)){
        
        this.innerBlocks = true;
        
          //get the current block to its end
        initial = i;
        id = block.get(i);
        int fist = id.indexOf(begin)+1;
        int last = id.indexOf(end)-1;
        id = id.substring(fist, last);
        boolean endFound = false;
        
        for(;i<block.size()-1;i++){

          if(block.get(i).contains(close + id + end)){

            String[] data = (String[]) this.block.subList(initial, i).toArray();
            ResBlock temp = new ResBlock(data,id);
            this.subBlocks.add(temp);
            initial = i;
            endFound = true;
          }//find closing tag
          
        }//end loop
        
        if(!endFound){
          throw new SlickException("Error parsing resource, end tag not found in: " + 
              this.identity + " section : " + id);
        }//check if resource is complete

      }//end if id found

    }//end parsing loop
    
  }//end constructor
  
  public boolean containsSubBlock(){
    
    return this.innerBlocks;
    
  }//end contains sub blocks
  
  public ResBlock getBlock(int index) throws SlickException{
    
    this.last = index;
    if (innerBlocks) {
      
      if (index < this.subBlocks.size()) {
        return this.subBlocks.get(index);
      }
      throw new SlickException("Error in resource block " + this.identity + " index out of bounds");
      
    }
    throw new SlickException("Error in resource block " + this.identity + " subBlocks is null");
    
  }//end get block by index
  
  public String getIdentity(){
    
    return this.identity;
  
  }//end get identity
  
  public ResBlock getNextBlock() throws SlickException{
    
    if(this.innerBlocks){
      
      int temp = last++;
      if (temp < this.subBlocks.size()) {
        return this.subBlocks.get(temp);
      }
      
    }
    
    throw new SlickException("Error in resource block " + this.identity + " next subBlock does not exist.");
    
  }//end get next block
  
  public boolean hasNext() throws SlickException{
    
    if (innerBlocks) {
      return (this.last + 1 < this.subBlocks.size());
    }
    throw new SlickException("Error in resource block " + this.identity + " next subBlock does not exist.");
    
  }//end has next block
  
  

}//end resource block

package midnight.util;

import java.util.ArrayList;

import midnight.util.resources.Resource;

import org.newdawn.slick.SlickException;


public class ResourceManager {
  
    //consider changing id's to type long
  protected static int id=0;
  protected static ArrayList<Resource> RESOURCES;
  
  public ResourceManager(){
    //figure it out later
    ResourceManager.RESOURCES = new ArrayList<Resource>();
  }//end constructor
  
  public ResourceManager(ArrayList<Resource> res){
    //figure it out later
    ResourceManager.RESOURCES = res;
  }//end constructor
  
  public static void addResource(Resource res){
    
    RESOURCES.add(res);
    
  }//end add resrouce single
  
  public static void addResource(ArrayList<Resource> res){
    
    for(Resource r : res){
      RESOURCES.add(r);
    }
    
  }//end add resource multiple
  
  public static int getNextID(){
    
    return id++;
    
  }//end get next id
  
  public static Object getResourceById(int id) throws SlickException{
    
    for(Resource r : RESOURCES){
      if(r.id == id){ return r.getResource(); }
    }
    throw new SlickException("Failed to locate resource #" + id);
    
  }//end get resource by id
  
  //public 

}//end resource manager





public class Resource {
  
  public final int id;
  private final Object resource;
  
  Resource(int id, Object res){
    
    this.id = id;
    this.resource = res;
    
  }//end constructor
  
  public Object getResource(){
    
    return this.resource;
    
  }//end get resource
  
}//end resource

 

[su root]

If you aren't going to use established XML libraries, I definitely wouldn't use XML at all.

Though before I create my own file format, I would look at other established formats (eg: JSON, JSO, YAML, INI files, etc).

Ultimately, your goal should be either A) easiest to edit for the user, or B) easiest to convert between an object in memory and a stored file version and vise-versa (i.e. easiest to save/restore).

 

[ssjwizard]

Yes its not actual XML its my own script that ill be parsing, its just styled similar to XML. As for ease of the user modifying them they wont be. This is meerly a text interface between game resources and the game world. The text files will be created by a resource editor GUI that im building.

 

[dropadrop]

Just a few notes...

I'm not too familiar with XML. I have written even a kludge parser of our own, though it's not very generic and as such won't add a lot of value to this... Well one thing I did not find from most online instructions was to check that a node was an element node:

if (child.getNodeType() == Node.ELEMENT_NODE) {
                        Element element = (Element)child;
                    }

This is a bigger problem with files that users have edited and looping though them...

But just a few pointers:

- I don't think your resourceManager will scale very well if all game resources are loaded into memory
- If you are going to support searching by ID (at least if it's common), you might want to use a map instead of a list
- I'm not finding the Block / ResBlock structure extremely intuitive, I'm probably missing something, but could'nt you just do something like

Element -> String name, Element previous, Element next, Element firstChild, Object data

In the example of your first post, [resource] would be an element with a link to the first child. It would probably not have any data of it's own. If you are just searching for something you can navigate by going to firstChild and from there go to next element etc. You can add methods to make things easier when you need them (get all children etc), but try to make the data structure simple and efficient.

Will you also be adding new blocks?

Sorry, probably not a very clear post

 

[ssjwizard]

if (child.getNodeType() == Node.ELEMENT_NODE) {
                        Element element = (Element)child;
                    }

- I don't think your resourceManager will scale very well if all game resources are loaded into memory

- If you are going to support searching by ID (at least if it's common), you might want to use a map instead of a list

- I'm not finding the Block / ResBlock structure extremely intuitive

Will you also be adding new blocks?

First thanks for the input!

#1 I think I see where your going with that check, Ill keep it in mind.

#2 All game resources will be loaded into memory for this particular game, its a session based map builder RPG type of game there will only be ~30 unique blocks of resources for each playset. All other resources are created from these master resources, IE the map blocks are all on one sprite sheet which gets chopped up into subimages, this leaves all of the color data in the originating image wtihout copying it so effectively a new resource only cost the memory space of its points, and other various bits.

#3 I do support searching by ID but it should only happen if a resource has gone missing, and during initialization.

#4 yes I agree the block system still needs some work, I had to put them down for now as my attention has been called away to a bug in the underlying engine that needs to be fixed ASAP.

#5 Im not sure what you mean? More types of blocks or adding new blocks while the game is running?

 

[dropadrop]

#5 Im not sure what you mean? More types of blocks or adding new blocks while the game is running?

Yeah, I'm just thinking of how the files will be modified. Will changing a resource file always require restarting the services? You mention a gui based tool, will it use other classes for accessing the files or are you extending these same ones?

 

[ssjwizard]

Ah ok, so there will be a set of files for each playset(urban, military base, graveyard, etc). The GUI tools are for my use only unless I release them as modding tools later. The files are only modified during the development stage, not something that the user will have unencrypted access to, and especially not something that could change during a match.