XCAT_REST_API_enhancements

{{:Design Warning}}

Problems

The current xcat rest api has several deficiencies that need correcting:

• not all xcat functionality is covered, or even the most commonly used function
• the documentation was poor and incomplete
• the code was disorganized and not commented
• the implementation is slow, it doesn't always use the best xml cmd for the function
• the input and output data structures follow the current xml poor and inconsistent structure

Enhancements

This is the list of what should be done to correct these problems (in approximate priority order):

• test & document all current api calls
  • the document needs to include input and output for all calls (bruce and bai yuan)
  • there should be an automated test driver in both curl (xcatws-test.sh) and native perl (xcatws-test.pl) so that improvements to xcatws.cgi can be quickly and easily tested (bai yuan)
  • test, fix, document the other resource handlers in there: groups, logs, notifications, policies, accounts, hypervisor(bai yuan)
• (done) organize/clean up code and add debugging (bruce)
• change xdsh & xdcp to be a post instead of put, and make the code more compact (bai yuan)
  • instead of checking for each individual flag, have a hash that maps keyword to flag, and then just process them all in a loop. Do this for the other api calls too. Since xcatws.cgi is loaded/processed by apache on every api call, if we let it get super big, it will slow each call down.
  • don't have to do it right away, but we might want to eventually break the different resource handlers (and corresponding json wrapper functions) into separate pm files, one for each resource. Each api call only references one resource, so only 1 pm would have to be loaded.
• change json input structure of all put and post calls, and change the json output structure of all get calls (bruce and bai yuan)
  • the put and post calls weren't using json correctly - most of them just had an array of "attr=val" strings. That should be a json object instead (equivalent to a perl hash). I fixed the node attribute change call and node create as examples. The rest of the put/post calls should be patterned after those.
  • the json output of the get calls just follows the same kind of structure as the xml output of xcatd, which has never been good (it is inconsistent and structured stangely). Even though it will make the get api calls a little slower (because of more processing), the wrapJson() function should restructure the output to the correct format for the json option. I've done the get node attributes, power, vitals, inventory, and energy as examples. The others should be patterned after that.
  • i'm using the openstack api as a model (mostly)
  • i think the json output format should be the primary one that we document and tell people to use. So just leave the formatting of the xml and html output the way it is now (don't bother improving it).
  • once the input and output structure is corrected for each call the corresponding documentation should be corrected
• change the implementation of the table api calls to use lissa's xml interface that she did for pcm (its faster)
  • document them as a faster/better alternative to the get api calls for each resource the use lsdef
  • here's info from lissa about her api: "the drivers that show how to use it are on 9.114.34.44 in /root/lissa/PCM/api. The driver I use is pcmdriver. All the XML input files are in there. I think they match the lastest code. I think the files that start with PCM* were actually given to me by PCM so I could see exactly how I was called. I don't see them in our release, so maybe they add them later. If you look at pcmdriver, it is pretty simple. Also the interface is commented at the top of each routine in tabutils.pm. The number files like getTablesAllNodeAttribsreq1 are just variations on the call like multiple table vs 1 table. Uncomment # $ENV{'XCATXMLTRACE'}=1; in pcmdriver and you will see the XML"
• add api calls for functionality that is missing (bai yuan)
  • osimage def create, change, delete and copycds
  • nodeset stat (to query current nodeset)
• eliminate the need to put the pw in the url (bai yuan)
  • use an api key or certificate instead. See what openstack does.
• figure out if the data sent back is given the correct Content/Type (bai yuan)
• compare the xcatws.cgi code that talks to xcatd (sendRequest()) to the Client.pm code to make sure it isn't missing any bug fixes in the last couple years (bai yuan)
• investigate trying to send output back to rest api client as it is coming from xcatd (instead of waiting for all of the data from xcatd before sending anything to the rest api client) (bai yuan)
  • some api calls can take a long time, especially for a lot of nodes, so to see nothing until the call is complete is not very good
  • this might be tricky, maybe only do it for html or xml output?
• do performance/scale tests with lots of nodes and optimize code (bai yuan)
• add api calls to return meta-data of resources (list of possible attributes of def objects)(bai yuan)

Other Design Considerations

• Required reviewers:
• Required approvers: Bruce Potter
• Database schema changes: N/A
• Affect on other components: N/A
• External interface changes, documentation, and usability issues: N/A
• Packaging, installation, dependencies: N/A
• Portability and platforms (HW/SW) supported: N/A
• Performance and scaling considerations: N/A
• Migration and coexistence: N/A
• Serviceability: N/A
• Security: N/A
• NLS and accessibility: N/A
• Invention protection: N/A