LuaStepper

20 August 2019 Link

On This Page



Description

LuaStepper is a module to help lua programs control simultaneous execution of multiple Lua threads without them needing to be coroutines. So it is like pre-emptive multitasking but instead of the operating system controlling the task switch and time slicing the Lua parent process does that with the API of this module.

Dependencies

LuaStepper has no dependencies. It just needs Lua >= 5.1

Installation and source code

Compile LuaStepper.c in the repository at Github and then copy the library module LuaStepper.dll/so to package.cpath directory and lstepper.lua to package.path directory. Or use LuaRocks to install it like:
> luarocks install luastepper

Basic Usage

The demo.lua file is in the test subdirectory of the source on Github. Once the module is installed the demo file can be directly run. The file is listed below:
  1. lStepper = require("lStepper")
  2. print("lStepper version is:",lStepper._VERSION)
  3.  
  4. -- Define script 1
  5. script1 = [[
  6. while true do 
  7.     print('A') 
  8. end]]
  9. -- Add script 1 to the queue with some print statements that are executed immediately
  10. print("Add script 1. ID is -->",lStepper.addTask(script1,"print('Initialization step 1') print('Initialization step 2')"))
  11.  
  12. -- Define script 2
  13. script2 = "while true do print('B') end"
  14. -- Add script 2 to the queue with some print statements that are executed immediately
  15. print("Add script 2. ID is -->",lStepper.addTask(script2,"print('Initialization step x') print('Initialization step y')"))
  16.  
  17. -- Define script 3
  18. script3 = [[
  19.             test1='Hello' 
  20.             test2=23 
  21.             test3=34.45 
  22.             test10 = {Hello=test1}
  23.             while(true) do 
  24.                 print('script3 test5=',test5) 
  25.                 if test4 and test4==test3 then
  26.                     break 
  27.                 end 
  28.             end 
  29.             while(true) do
  30.                 print('script3 next loop') 
  31.                 if test5 and type(test5)=='table' and test5.test6 then 
  32.                     print('test6=',test5.test6) 
  33.                     break 
  34.                 end 
  35.             end]]
  36. -- Add script 3 to the queue with a print statement initialization and a variable setting that is then printed in the script
  37. print("Add script 3. ID is -->",lStepper.addTask(script3,"print('Adding Script 3') test5='script3'",5,true))    -- 5 means the number os codes to execute for each step of this script
  38.                                                                                         -- This script is also placed in the urgent task pile by setting true
  39. print("Number of tasks=",lStepper.getNumOfTasks())    -- displays 2   1   -- 2 regular 1 urgent
  40. print("Start stepper loop")
  41. for i=1,50 do
  42.     x,y=lStepper.runLoop()    -- Run all the tasks for 1 step
  43.     if not x then
  44.         print(y)
  45.         break
  46.     end
  47.     if((x[0][1] == "FIN" or x[0][1]:sub(1,2) == "NE") and (x[1][1] == "FIN" or x[1][1]:sub(1,2) == "NE")) then
  48.         print("Both scripts ended")
  49.         break
  50.     end
  51.     if i==5 then
  52.         print("i=5, script 1 status is ",lStepper.taskStatus(0))
  53.         print("i=5, script 2 status is ",lStepper.taskStatus(1))
  54.     end
  55.     if i==30 then
  56.         print("i=30, script 1 status is ",lStepper.taskStatus(0))
  57.         print("i=30, script 2 status is ",lStepper.taskStatus(1))
  58.     end
  59.     if i==20 then
  60.         print("B suspended")
  61.         lStepper.suspendTask(1)
  62.     end
  63.     if i==40 then
  64.         print("B resumed")
  65.         lStepper.resumeTask(1)
  66.     end
  67.     if i==5 then
  68.         x,y = lStepper.getTaskData(4,"test3")    -- Get the value of test3 from script 3
  69.         if x then
  70.             print("Value from script3 test3=",x)
  71.             print(lStepper.setTaskData(4,{test4=x}))    -- Set value of test4 to be same as test3 in script 3
  72.         else
  73.             print("Error getting data:",y)
  74.         end
  75.     end
  76.     if i==10 then
  77.         local t = {
  78.             test6='Added Table!'
  79.         }
  80.         print(lStepper.setTaskData(4,{test5=t}))
  81.         x=lStepper.getTaskData(4,"test10")
  82.         print(x)
  83.         if type(x) == 'table' then
  84.             print("test10 in script 3 is a table:")
  85.             for k,v in pairs(x) do
  86.                 print(k,v)
  87.             end
  88.         end
  89.     end
  90. end
  91.  
  92. print("END",lStepper.taskStatus(0))
  93. print("END",lStepper.taskStatus(1))
  94. print("END",lStepper.taskStatus(4))
  95. lStepper.closeTask(0)
  96. lStepper.closeTask(1)
  97. lStepper.closeTask(4)


The demo loads 2 scripts to run in parallel and runs them. It suspends 1 script for some time and then resumes it. It also shows how to run initialization code when the script is loaded and how to read and pass data to a script.

lStepper Lua module vs LuaStepper C module

The demo shows that you just need to require lStepper which is the interface to the LuaStepper C module written in Lua. The LuaStepper C module can also be used directly but lStepper makes it easier to use the C module without any loss of functionality. lStepper requires LuaStepper and uses its API.

Thread creation details

  • Each thread environment contains all standard lua libraries supplied with Lua except debug, package, io and os libraries. These libraries are excluded since they can make the thread get stuck so alternatives can be provided when needed. Although if you want to provide them you can use the runCode function described in the API below.
  • The environment has a table called _LS, which has the following functions and objects:
    • tableToString: FUNCTION : Function to convert a table to a string. Keys can be a number, string or table, values can be number, string table or boolean. Recursive tables are handled but metatables are not handled. This function can be used to serialize and communicate data with the parent state. This is the function that is used by the LuaStepper API to communicate data to the controller state.
  • Each thread runs in its separate Lua state

API Reference

lStepper

The API provided by the lua interface is accessed by requiring 'lStepper'. This is the recommended way to access the LuaStepper module. 
  1. lStepper = require("lStepper")


API

addTask

This adds a task to the task list. All the tasks in the task list are stepped through with the indicated number of steps when the runLoop function is executed.
Syntax:
addTask((script: string)[, (Initialization code: string)][, (number of steps: number)][, (urgent pile: boolean)]) -> (script index: number)
Usage Example:
  1. script = "while true do print('B') end"
  2. scriptidx,msg = lStepper.addTask(script,"print('Adding Script')",5,true)

Inputs:
  • script - is the string containing the Lua script that is the task
  • Initialization code - is an optional Lua script as a string which is executed without stepping before the actual code starts running in the environment.
  • number of steps - is the optional number of steps to step the script each time runLoop executes a loop. These number of steps may not correspond to the lines of code.
  • urgent pile - is an optional boolean indicating whether this task has to be placed in the urgent quota.
    NOTE: Urgent tasks are placed with task Indices beginning after the main task indices. This facility is provided to have an extra buffer to add urgent tasks without removing the normal tasks. Recommended way to use them would be add a urgent task. Suspend all the main tasks and then run the urgent task to get something done and then clean the urgent task slot and then resume the main tasks again.
Returns:
  • nil and a error message on failure
  • task index if successful

runLoop

This runs each task in the task List with their respective indicated number of steps.
Syntax:
runLoop( [(number of runs: number)] ) ->(tasks status: table)
Usage Example:
  1. stat,msg=lStepper.runLoop()

Inputs:
  • number of runs - is an integer which specifies the number of times the loop has to run, by default it is run just once.
Returns:
  • nil if there are no tasks
  • nil and error message if failure
  • table with keys as the task index and the values are tables with at most 2 elements representing the status of each task in the pile. The 1st element of each status table can be:
    • GOOD - Task is running good
    • FIN - Task Finished
    • ERROR - Task produced an error and stopped. The second element is the error message
    • NE-PREVERR - Task was not executed because it had a previous error. Second element is the Error code type i.e."LUA_ERRRUN" or "LUA_ERRMEM"
    • NE-FIN - Task was not executed because it finished already
    • NE-SUSPEND - Task was not executed because it was suspended before
    In case of multiple loop runs the status is what was generated in the last iteration.

suspendTask

Suspends a task's execution till resumed by calling resumeTask.
Syntax:
suspendTask( (task index: number) ) -> (true: boolean)
Usage Example:
  1. stat,msg = lStepper.suspendTask(1)

Inputs:
  • task index- index of the task that needs to be suspended
Returns:
  • nil and a error message in case of failure
  • nil if the task did not exist or had already finished or suspended.
  • true if suspend was successful

resumeTask

Resumes a suspended task's execution.
Syntax:
resumeTask( (task index: number) ) -> (true: boolean)
Usage Example:
  1. stat,msg = lStepper.resumeTask(1)

Inputs:
  • task index- index of the task that needs to be resumed
Returns:
  • nil and a error message in case of failure
  • nil if the task did not exist or was not suspended or was finished.
  • true if resume was successful

taskStatus

To poll the status of a task
Syntax:
taskStatus( (task index: number) ) -> (status: string)[, (status detail: string)
Usage Example:
  1. stat,msg = lStepper.taskStatus(1)

Inputs:
  • task index - index of the task whose status is being queried
Returns:
  • nil and a error message in case of failure
  • GOOD - Task is running good
  • SUSPENDED - Task is in suspended state and is not being executed when runLoop is called
  • NE-PREVERR - Task exited with an error when it was last executed. Second return is the Error code type i.e."LUA_ERRRUN" or "LUA_ERRMEM"
  • NE-FIN - Task finished already

setTaskData

To pass data to a task's environment
Syntax:
setTaskData( (task index: number), (data table: table) ) -> (true: boolean)
Usage Example:
  1. stat,msg = lStepper.setTaskData(4,{test5=30})

Inputs:
  • task index - index of the task to which data has to be passed
  • data table - table containing key value pairs that have to be set in the task's Lua environment. It would overwrite same keys all values in the Global environment of the task. The keys can be number or string while the values can be string, number, boolean or table
Returns:
  • nil and a error message if error
  • true if successful

getTaskData

To read data from a task's environment
Syntax:
getTaskData( (task index: number), (variable name: string) ) -> (value: any type)
Usage Example:
  1. val,msg = lStepper.getTaskData(4,"test10")

Inputs:
  • task index - index of the task to which the table has to be passed
  • variable name - name of the variable in the task's global environment whose value is to be retrieved
Returns:
  • nil and a error message if error
  • the value if success

getNumOfTasks

To return the number of tasks in the tasks list
Syntax:
getNumOfTasks() -> (tasks in regular list: number), (tasks in urgent list: number)
Usage Example:
  1. normal,urgent = lStepper.getNumOfTasks()

Returns:
  • nil and error message if error
  • number of tasks in the task List as the 1st return and number of urgent tasks in the list as the 2nd return

closeTask

The close the task and free up its index in the list
Syntax:
closeTask( (task index: number) ) -> (true: boolean)
Usage Example:
  1. stat,msg = lStepper.closeTask(0)

Inputs:
  • task index - index of the task which is to be closed
Returns:
  • nil if task was not there at the index
  • nil and error message if error
  • true if successful

registerCallBack

To register a callback function with LuaStepper. This function is accessible to any task's runCode function and other function it creates by calling lstep.callParentFunc. That means you can call this registered function by doing the following:
  1. lStepper.runCode("lstep.callParentFunc(arg1,arg2)")

So this allows passing arguments directly to your parent. Another way could be exposing a function to the thread which can then in turn call the call back function. So we could have:
  1. lStepper.runCode([[
  2.     newFunc = function(arg)
  3.          result = lstep.callParentFunc("newFunc",arg)
  4.     end
  5. ]])

Here runCode exposed a function called newFunc to the thread code which can pass data to the call back function and also tells the call back function about its own name so the call back function can detect where the call came from and take appropriate action and return the result.
The Call back function
The call back function is called with the 1st argument as the task index followed by all the arguments passed to the call back function in the task. So in the above case the call back function call would be something like this:
  1. callback(1,"newFunc",arg)  -- 1 is the task index of this task

The second argument being the function name is not enforced but should be followed since when the task calls require the call back function is called with its second argument "require". So following this standard would lead to consistent call back function programming.
The call back function can return 2 values. Both may be strings, the 1st can be nil.

By default the require function exposed to the task's code uses the call back function to allow exposing external code to the thread. So if you don't have any call back registered in LuaStepper then require won't work in any task's code. So when the thread code calls require it calls this call back function to tell the parent thread of the require request.
Syntax:
registerCallBack( (callback: function) ) -> (true: boolean)
Usage Example:
  1. stat,msg = registerCallBack(myCallBackFunc)

Inputs:
  • callback - Call back function to be exposed to all thread's runCode function
Returns:
  • nil and error message if error
  • true if success

runCode

Allows the parent thread to run some code in the thread environment without doing the stepping. This is useful to provide resources as requested by the thread in the thread environment.
Syntax:
runCode( (task index: number), (Lua Code: string) ) -> (true: boolean)
Usage Example:
  1. stat,msg = lStepper.runCode(1,[[x = 2+3]])

Inputs:
  • task index - index of the task where the code needs to be run
  • Lua Code - Lua code script to run
Returns:
  • nil and error message if error
  • true if success
Notes:
The environment in which the code runs is the global environment of the thread plus it has access to the following extra functions/tables which the thread cannot access:
    • pack table - containing the original package table
    • req function - containing the original require function
    • lstep table - table containing special functions used by the thread management functions. It is only accessible via runCode or with functions provided to the task by runCode. Its members are:
      • callParentFunc - Function that calls the function that is registered by the parent thread using registerCallBack
      • getswitchstatus - which toggles between -1 and 1 for every switch that LuaStepper does away from the task. This can be used by the thread management functions to detect a switch.
    • oslib library - containing the original os library
    • iolib library - containing the original io library
Thus if you want to expose the os library to your thread then simple run the following code:
  1. lStepper.runCode("os = oslib")


Objects

_VERSION

Returns the version number of the module

LuaStepper

The API provided by the LuaStepper C module is accessed by requiring 'LuaStepper'. Although this can be used directly the Lua interface is the preferred way of accessing the LuaStepper module.
  1. LuaStepper = require("LuaStepper")

API

The API functions are as follows:

addTask

This adds a task to the task list. All the tasks in the task list are stepped through with the indicated number of steps when the runLoop function is executed.
Syntax:
addTask((script: string)[, (Initialization code: string)][, (number of steps: number)][, (urgent pile: boolean)]) -> (script index: number)
Usage Example:
  1. script = "while true do print('B') end"
  2. scriptidx,msg = LuaStepper.addTask(script,"print('Adding Script')",5,true)

Inputs:
  • script - is the string containing the Lua script that is the task
  • Initialization code - is an optional Lua script as a string which is executed without stepping before the actual code starts running in the environment.
  • number of steps - is the optional number of steps to step the script each time runLoop executes a loop. These number of steps may not correspond to the lines of code.
  • urgent pile - is an optional boolean indicating whether this task has to be placed in the urgent quota.
    NOTE: Urgent tasks are placed with task Indices beginning after the main task indices. This facility is provided to have an extra buffer to add urgent tasks without removing the normal tasks. Recommended way to use them would be add a urgent task. Suspend all the main tasks and then run the urgent task to get something done and then clean the urgent task slot and then resume the main tasks again.
Returns:
  • nil and a error message on failure
  • task index if successful

runLoop

This runs each task in the task List with their respective indicated number of steps.
Syntax:
runLoop( [(number of runs: number)] ) ->(tasks status: table)
Usage Example:
  1. stat,msg=LuaStepper.runLoop()

Inputs:
  • number of runs - is an integer which specifies the number of times the loop has to run, by default it is run just once.
Returns:
  • nil if there are no tasks
  • nil and error message if failure
  • table with keys as the task index and the values are tables with at most 2 elements representing the status of each task in the pile. The 1st element of each status table can be:
    • GOOD - Task is running good
    • FIN - Task Finished
    • ERROR - Task produced an error and stopped. The second element is the error message
    • NE-PREVERR - Task was not executed because it had a previous error. Second element is the Error code type i.e."LUA_ERRRUN" or "LUA_ERRMEM"
    • NE-FIN - Task was not executed because it finished already
    • NE-SUSPEND - Task was not executed because it was suspended before
    In case of multiple loop runs the status is what was generated in the last iteration.

suspendTask

Suspends a task's execution till resumed by calling resumeTask.
Syntax:
suspendTask( (task index: number) ) -> (true: boolean)
Usage Example:
  1. stat,msg = LuaStepper.suspendTask(1)

Inputs:
  • task index- index of the task that needs to be suspended
Returns:
  • nil and a error message in case of failure
  • nil if the task did not exist or had already finished or suspended.
  • true if suspend was successful

resumeTask

Resumes a suspended task's execution.
Syntax:
resumeTask( (task index: number) ) -> (true: boolean)
Usage Example:
  1. stat,msg = LuaStepper.resumeTask(1)

Inputs:
  • task index- index of the task that needs to be resumed
Returns:
  • nil and a error message in case of failure
  • nil if the task did not exist or was not suspended or was finished.
  • true if resume was successful

taskStatus

To poll the status of a task
Syntax:
taskStatus( (task index: number) ) -> (status: string)[, (status detail: string)
Usage Example:
  1. stat,msg = LuaStepper.taskStatus(1)

Inputs:
  • task index - index of the task whose status is being queried
Returns:
  • nil and a error message in case of failure
  • GOOD - Task is running good
  • SUSPENDED - Task is in suspended state and is not being executed when runLoop is called
  • NE-PREVERR - Task exited with an error when it was last executed. Second return is the Error code type i.e."LUA_ERRRUN" or "LUA_ERRMEM"
  • NE-FIN - Task finished already

setTaskData

To pass data to a task's environment
Syntax:
setTaskData( (task index: number), (data table: table) ) -> (true: boolean)
Usage Example:
  1. stat,msg = LuaStepper.setTaskData(4,{test5=30})

Inputs:
  • task index - index of the task to which data has to be passed
  • data table - table containing key value pairs that have to be set in the task's Lua environment. It would overwrite same keys all values in the Global environment of the task. The keys can be number or string while the values can be string, number, boolean.
Returns:
  • nil and a error message if error
  • true if successful

setTaskTable

To pass a table to a task's environment. Note this function is not present in lStepper module since the function setTaskData there combines the functionality of this function as well.
Syntax:
setTaskTable( (task index: number), (table name: string), (table script: string) ) -> (true: boolean)
Usage Example:
  1. stat,msg = LuaStepper.setTaskTable(4,"newTable","t0={1,2,3}")

Inputs:
  • task index - index of the task to which the table has to be passed
  • table name - is the name by which the table will be referenced in the task's environment
  • table script - a Lua script as a string which when executed will generate the table in the value t0 in the script.
Returns:
  • nil and a error message in case of error
  • true if success.

getTaskData

To read data from a task's environment
Syntax:
getTaskData( (task index: number), (variable name: string) ) -> (value: any type)[, ("TABLE": for value of table)]
Usage Example:
  1. val,msg = LuaStepper.getTaskData(4,"test10")

Inputs:
  • task index - index of the task to which the table has to be passed
  • variable name - name of the variable in the task's global environment whose value is to be retrieved
Returns:
  • nil and a error message if error
  • the value if the value was of the type string, number, boolean
  • the table as a table generation script string and "TABLE" if the value was a table. To convert the string to the table simply load and execute the string and take the value of 't0' from the execution environment.

getNumOfTasks

To return the number of tasks in the tasks list
Syntax:
getNumOfTasks() -> (tasks in regular list: number), (tasks in urgent list: number)
Usage Example:
  1. normal,urgent = LuaStepper.getNumOfTasks()

Returns:
  • nil and error message if error
  • number of tasks in the task List as the 1st return and number of urgent tasks in the list as the 2nd return

closeTask

The close the task and free up its index in the list
Syntax:
closeTask( (task index: number) ) -> (true: boolean)
Usage Example:
  1. stat,msg = LuaStepper.closeTask(0)

Inputs:
  • task index - index of the task which is to be closed
Returns:
  • nil if task was not there at the index
  • nil and error message if error
  • true if successful

registerCallBack

To register a callback function with LuaStepper. This function is accessible to any task's runCode function and other function it creates by calling lstep.callParentFunc. That means you can call this registered function by doing the following:
  1. LuaStepper.runCode("lstep.callParentFunc(arg1,arg2)")

So this allows passing arguments directly to your parent. Another way could be exposing a function to the thread which can then in turn call the call back function. So we could have:
  1. LuaStepper.runCode([[
  2.     newFunc = function(arg)
  3.          result = lstep.callParentFunc("newFunc",arg)
  4.     end
  5. ]])

Here runCode exposed a function called newFunc to the thread code which can pass data to the call back function and also tells the call back function about its own name so the call back function can detect where the call came from and take appropriate action and return the result.
The Call back function
The call back function is called with the 1st argument as the task index followed by all the arguments passed to the call back function in the task. So in the above case the call back function call would be something like this:
  1. callback(1,"newFunc",arg)  -- 1 is the task index of this task

The second argument being the function name is not enforced but should be followed since when the task calls require the call back function is called with its second argument "require". So following this standard would lead to consistent call back function programming.
The call back function can return 2 values. Both may be strings, the 1st can be nil.

By default the require function exposed to the task's code uses the call back function to allow exposing external code to the thread. So if you don't have any call back registered in LuaStepper then require won't work in any task's code. So when the thread code calls require it calls this call back function to tell the parent thread of the require request.
Syntax:
registerCallBack( (callback: function) ) -> (true: boolean)
Usage Example:
  1. stat,msg = registerCallBack(myCallBackFunc)

Inputs:
  • callback - Call back function to be exposed to all thread's runCode function
Returns:
  • nil and error message if error
  • true if success

runCode

Allows the parent thread to run some code in the thread environment without doing the stepping. This is useful to provide resources as requested by the thread in the thread environment.
Syntax:
runCode( (task index: number), (Lua Code: string) ) -> (true: boolean)
Usage Example:
  1. stat,msg = LuaStepper.runCode(1,[[x = 2+3]])

Inputs:
  • task index - index of the task where the code needs to be run
  • Lua Code - Lua code script to run
Returns:
  • nil and error message if error
  • true if success
Notes:
The environment in which the code runs is the global environment of the thread plus it has access to the following extra functions/tables which the thread cannot access:
    • pack table - containing the original package table
    • req function - containing the original require function
    • lstep table - table containing special functions used by the thread management functions. It is only accessible via runCode or with functions provided to the task by runCode. Its members are:
      • callParentFunc - Function that calls the function that is registered by the parent thread using registerCallBack
      • getswitchstatus - which toggles between -1 and 1 for every switch that LuaStepper does away from the task. This can be used by the thread management functions to detect a switch.
    • oslib library - containing the original os library
    • iolib library - containing the original io library
Thus if you want to expose the os library to your thread then simple run the following code:
  1. LuaStepper.runCode("os = oslib")


Objects

_VERSION

Returns the version number of the module

To Do

  1. Add function for utility code to yield the thread it is running in
  2. Implement coroutine.wrap in the thread code.