L - A Scripting Language Proposal

L is based on simple ideas that can be built upon to create more powerful ones. Unlike Smalltalk, which strives to do the same, it is based on fairly familiar language constructs. One does not have to build their own IF construct, for example.

The most striking features of L are:

• Only one type: a hybrid/dual string and associative array

• Associative array used for multiple purposes and paradigms

• Caters to multiple syntax styles

• No built-in "nested data structures"

• Libraries provide collections and not built-in syntax

Note that although L makes heavy use of associative arrays (A.k.a. dictionaries), I intend them mostly as interface mechanisms and not data storage nor collection manipulation. I assume that a table-based API will handle most data needs instead. (I am still working on the table/collection API.) In this sense L is a lot like Smalltalk in that collection manipulation is via libraries and not dedicated language syntax. This keeps the language simpler and more generic.

That reflects one important general philosophy of L: provide the syntax mechanisms to handle collections, but don't make the collection handling part of the built-in syntax.

Blocking and semi-colons

  sub mysub(x)   // endx version of routine
     if x %n> 3   // we will explain the "n" later
        out "it's greater than 3"
        y = 1 + 2 + 3 + 4 _    // continued line
        + 5 + 6
     endIF
  endSub

  sub mysub(x) {   // brace style version 
     if x %n> 3 {
        out("it's greater than 3");
        y = 1 + 2 + 3 + 4    // line is split
        + 5 + 6;
     }
  }

I have not yet determined how style preference is indicated. One approach is the have a "setting _braces" or "setting _endx" at the top to indicate style (endx being the default), or use auto-detection where the first character after the routine declaration or block determines the interpretation for that routine/block. The technique selected will depend on interpreter technology feasibility. It might also be that an implementor only chooses one style or the other based on audience. Perhaps it can even be a dynamic setting, accessible via the "sys" array.

  mysub x, y + 23        // valid in endx style
  a = mysub(x, y + 23)   // valid
  a = mysub x, y + 23    // invalid

L allows named parameters as well as positional parameters. Positional parameters must come before any named parameters.

  mysub(1, 2  _waiton _where j > 3 _orderby lname)
  // See "note on symbols" below for "#" explanation
  mysub(1, 2  #waiton #where j > 3 #orderby lname)  
  mysub 1, 2  _waiton _where j > 3 _orderby lname
  mysub 1, 2  _waiton _where "j > 3" _orderby "lname"
  result = mysub(1, 2 _waiton _where j > 3 _orderby lname)

The arguments are always passed in as strings. (Extracting parameters will be discussed later). Thus, "lname" above is a string and not an explicit variable. (It could perhaps be interpreted as a variable in the called routine, but this is purely up to the called routine.)

This may sound messy at first, but by not allowing the "leaky assignments" found in many scripting languages, (such as x = y = z = a), most confusion is immediately eliminated. Also note that a comma after the "2" parameter in the above example is optional. I figured if I picked one or the other, then people would keep getting it confused. The interpreter would generally use the first underscore encountered to know where the named parameters start. Also note that the comparisons shown above are assumed to be passed to SQL, since they are not legal in L. Named parameters are assumed to mostly be used by frameworks, and not casual programmer routines since they take longer to learn how to write.

Named parameters make creating database-friendly and phrase-based syntax easier. One can emulate say XBase or Hypertalk syntax with this. It also provides "message-oriented" syntax, such as found in Smalltalk. Named arguments are very useful in cases where there may be many options, but only a few deviate from the default in practice.

A Note on Symbols

Here are the primary symbols used for our examples, and only for examples:

_   Named parameter indicator

$   Scope specifier (see note)

#   Assignment expression repeater

@   Write-enabled parameters

%   Comparitor marker

&   String concatenation

Note that the dollar sign is also used for embedding variables into strings. However, since they are inside of strings, they may be considered in a different category. Also note that variable embedding is only supported in some routines. In our case, we are assuming that Out() and OutLn() translate them. The string concatenation character "&" perhaps should be also put on the auction block, however, nobody seemed to complain about that choice.

Types and Strings

L has only one type. However, for familiarity purposes, you can pretend that it has 2 types: strings and associative arrays. (See Misc. Notes about the one type.) This may seem restrictive at first, but actually it is liberating. First, let's look at uses for strings.

What "type" something is depends completely on the context that you use it in. It is somewhat like Perl in that regard. This approach requires that some operators be clear about how they will act.

  a = "5"
  b = 2
  out a + b   // result: 7
  out a & b   // result: 52

Comparisons are done by putting a "comparative operator" next to the relationship symbol.

  if a %t= b     // compare as text
  if a %n= b     // compare as numeric
  if a %g= b     // general compare (see below)
  if a %tx= b    // no trimming before comparing 
  if a %ta= b    // trim all white spaces before comparing
  if a %u= b     // compare as Unicode
  if a %tb= b    // ignore all white space, even between
  if a %tc= b    // case sensitive compare
  if a %i= b     // compare as integer (round at 0.5)
  if a %i> b     // greater than example with integers

All comparison operators need at least one indicator letter. Thus, if you accidentally use a non-L habit, you will get a syntax error. Also note that the default for text comparing is right trimming.

The "g" operator if for "general" comparing. It can be used for both text and strings. It acts as if it trims both sides, ignores case, and removes any leading zeros and any trailing decimal zeros (if any) before comparing. This is the default for case statements, by the way. (I have been debating whether to use an implied "g" if there is no comparison operator. I am on the fence with that one.)

Associative Arrays

L uses the multi-emulation method to cover multiple "array" dimensions.

  var myarray[]      // declare an array
  myarray[1] = 'X'
  myarray.1 = 'X'     // same as prior
  myarray[1,1] = 'X'   
  myarray["1,1"] = 'X'  // same as prior
  myarray["foo"] = 'X'
  myarray.foo = 'X'       // same as prior

L has something known as "hidden entries" (HE) in associative arrays. There are two kinds of hidden entries: reserved and user-defined.

  myarray["~sys_lang_numbers"] = 'yes'   // reserved
  slset(myarray,"numbers","yes")      // same as prior
  myarray["~sys_customx"] = "foo"   // user-defined
  set(myarray,"customx",'foo')     // same as prior

The reserved "trimming" setting, for example, causes L to remove leading zeros and/or spaces from the array key before inserting it or searching for it. The possible values are:

0 - No trimming (default)
1 - Rid any white space
2 - Rid white space and and trim leading zeros
(May be deprecated, see below)

Setting 2 would translate a key of "001" to "1". Commas are also looked at, so that "01, 01, 001" becomes "1,1,1". Spaces are also removed. Setting 2 is ideal for using numeric subscripts. (Remember that L values are not typed. Therefore, the 2 setting is only a trimming behavior. Strings can still be used.)

Thus, to set up an array for primarily numeric use:

  var foo[]
  slset(foo, "trimming", 2)
  numericate(foo)  // same as prior

I am thinking about using the letter comparitor operaters instead of the above numbers. It seems numbers are reinventing the wheel covered by the comparitor letters.

Another reserved HE is the "parent" setting. This will cause L to look up any missing keys in the array named in the "parent" key. ("~sys_lang_parent" is the full name.) This can be used for OO inheritance, as we will see later.

Reserved HE "onDone" ("~sys_lang_onDone") is used to execute a snippet of code when the array falls out of scope, or program end, whichever is first. This allows utilities written in L to automatically perform cleanup, such as closing a table. (More on table API's is presented below.)

Reserved HE "onChange" ("~sys_lang_onChange") is used to trigger custom operations when any value in an array is changed. For example, it can be used in a database API to determine whether or not the current record needs to be saved. (It's sister HE is "onRead".) Note that just because it is a "reserved" HE does not mean that it cannot be programmer-assigned. Being reserved only means that you cannot delete it.

The user defined HE's can be used for whatever purpose the programmer can envision.

Control Statements

  sub myroutine(x)
     while x %n< 7 
       x = x * 2 + 1
     endwhile

     for each a in b[]
        out a, b
     endfor

     for i = 1 to 7 step 1
       x = i - y
     end for

     select x
     case 1
       foo()
     case 2,3,4
       bar()
     otherwise
       foo()
     end select
     // case comparisons are based on
     // left-zero-trimmed and space-trimmed text.
     // If this is problematic for a situation,
     // then if-elseif-else can be used instead.
  endsub

  //------ Now, the curly braces version
  sub myroutine(x) {
     while x %n< 7 { 
       x = x * 2 + 1;
     }

     for each a in b[] {
       out(a,b);
     }

     for i = 1 to 7 step 1 {
       x = i - y;
     }

     for(i=1; i < 7; i++) {   // C-style for accepted
       x = i - y;
     }

     select x {
       case 1 {
         foo();
       }
       case 2,3,4 {
         bar();
       }
       otherwise {
         foo();
       }
     }
  }
  

"Exit for" and "exit while" allow one to exit the corresponding loop. The exiting applies to the innermost loop if there is a potential overlap.

Nulls

  if x = null then ...  // the old way

  if x t= "(null)" then ... // the new way

Scope

  global x = 4
  sub routine8
     var x = 50
     routine9()
  end sub
  sub routine9
     var x = 2
     out x       // result: 2
     out my$x    // result: 2
     out global$x  // result: 4
  endSub

L also supports "regional" variables. This means that a variable has file-level scope, or "region" scope if inside of a defined region.

  regional x
  x = 30
  sub foo()
     var x = 2
     out x     // result: 2
     out regional$x   // result: 30
  end sub
  // the "regional" qualifier would not be
  // needed if there was not a local x.

At the top of the file (near Setting clauses), other program files can be "recognized" using syntax like the following:

  recognize mycode.L

If there is a routine naming overlap, the local routine gets precedence. If there is no local routine of a given name, but it is defined in multiple files, then an error is raised. At this point, you may need to use the file name scope specifiers for such routines.

Region names can also be declared to create package-like scope:

region foo
// a bunch of statements and routines here
end region
...
foo$aroutine()   // "foo$" may be optional

L supports the inherit option to allow access to parent's scope. The "my$" scope specifier (see above) is more important if Inherit is used in order to distinguish between local and inherited variables. Without it, precedence goes to the caller. (If it did not, then we could not guarantee access to all scope ranges under the condition of name overlaps.)

I chose not to use a scope-specifier for parent variables so that routines can be split without adding specifiers to all the working variables. That is a pain.

  sub a
    var x = 4
    b()
  endSub
  sub b inherit
    c()
  endSub
  sub c inherit
    out x    // will print 4
  endSub

Parameter Processing

  x = "cat"
  foo "dog", x, _yaddle _daddle 12
  ...
  sub foo(za, @ma)
     ma = 4     // we changed it for caller
     for each i in args[]
        outln i & ": " & args[i]
        // could also: outln "$i: $args[i]"
     endFor
  endSub

Typical output:
1: dog
2: cat
yaddle: 
daddle: 12

All named parameters (which start with underscores here) are returned as keys in the "args" array, and any arguments to the named parameters as the values under the keys. You can use the Eval() function to evaluate them. You may want to use the Inherit keyword so that the parent's variables can be examined if variables are passed. Or, you can use the "caller$" scope prefix.

Note that the "arg" array will not contain another array if a positional parameter is an array, but only the "~sys_lang_value" entry. Remember, nested arrays are against L's philosophy.

The "@" symbol indicates that the value is changable (passed by reference). Parameters without "@" are read-only (roughly equivalent to pass-by-value).

Table API's

  s = "select * from prices"
  t = openDB( _sql s  _readonly)
  while getNext(t)   // display price of all items
     out   "Price of item " & t.descript 
     outln " is " & t.price
  endWhile
  setFilter t, _where price < 15.00
  ...
  close(t)

  while t.getnext()

  while eval(t.getnext)

Note that usage of function syntax for table handle operations and dot syntax for fields is the reverse of the Visual Basic Script (VBS) convention. In VBS you would say something like t("price") instead of t.price and t.getnext() instead of getnext(t). The L approach is preferable because fields are accessed more often then handle operations in most programs. It is rumored that VBS may eventually use "!" to distinguish them instead of functional syntax, as it does in VB. Perhaps L can use ! to indicate a hidden entry in future editions.

Set-based DB engines (for SQL) would probably get all selected records at once and buffer them, while cursor-oriented approaches (like XBase) would probably retrieve them from the DB as needed (with possible sub-buffering for speed). Note that the syntax for each approach would not change much except perhaps for DB driver selections on opening. This allows one to swap drivers without major rewrites of record handling application code.

The "onDone" HE reserved setting (described previously) can be useful for automatically closing a table when a handle falls out of scope. The previously described "onChange" HE could also be helpful for database engine implementation.

Object Oriented-Like Syntax

Note that I am not a fan of OO. However, I am allowing L to cater to OO in the hopes that it may help "sell" it, and allow the rebirth of table-oriented syntax and all the rest of the good things that L provides. Thus, like Steve Jobs' Apple deal with Microsoft, compromise is sometimes needed for survival.

In L, the equivalent of the structure is the associative array. Behavior is given to the elements by simply using the parenthesis syntax:

  // Example of indirection
  var foo[]
  foo.x = "outln 'Hello L World!'"
  outln foo.x   // result: outln 'Hello L World!'
  foo.x()       // result: Hello L World!
  eval(foo.x)   // equiv. to prior

Although being able to execute a value and visa verse may seem risky to some, this is a scripting language, where protection is not the primary concern. L in general uses context of usage rather than type stamping to determine what can be done to what. The benefit for sacrificing protection is added run-time flexibility.

These are already part of L. L's OO syntax simply borrows them. However, there are two features of L that were added for OO purposes (but could be used for other purposes).

The first is the class structure:

  class foo
     this.x = 23.4
     this.y = -12.9
     method aMethod 
       yaz(this.x)
       raz(this.y)
     endMethod
  end class

  global foo[]
  foo.x = 23.4
  foo.y = -12.9
  foo.aMethod = "yaz(this.x) \n raz(this.y)"

  canEval(foo.aMethod)

One drawback of this approach is that parameters cannot be directly passed to methods. One must set class values to pass parameters.

  mail.to = "bob@host.com"
  mail.from = "spammer@spammakers.com"
  mail.title = "You too can be rich!"
  mail.send()

  class tiger
    slset(this, "parent", "feline")
    ...
  end class

To "instantiate" a new object, use the Clone() function.

Although this has storage efficiency problems at large quantities because the code is duplicated for each instance, it is assumed that some sort of collection system or database is used to store large quantities of instance information, and not "objects." Note that the inheritance feature (parent HE) can sometimes be used alleviate this problem because it provides a path back to a single instance. L's instantiation-by-copy drawbacks somewhat resemble those of JavaScript.

Note that the "Inherit" keyword has nothing to do with OO.

Miscellaneous Notes

1. In L, reserved words cannot be used for function names nor variables. Further, there cannot be a variable and array of the same name (except possibly at different scope levels). One can optionally place empty brackets "[]" after an array name to clarify it's type if desired.

2. Either single or double quotes can be used as string delimiters. This allows doubles or singles to be used within a string without escaping them (L uses a backslash for escapes).

3. Predefined settings and Recognize clauses go at the very top of each program file. Only comments can come before. Also, multiple settings can be given per Settings clause:

         setting _braces _foo off _bar _explicit
   

4. The Explicit setting is sort of like VB's "option explicit". Under it, all variables must be declared with var, global, static etc. However, unlike VB, if this setting is off, new variables can only appear on the assignment end of a statement, not the reference side. In other words, you can implicitly declare a variable by assigning a value to it, but not by referencing it (if explicit is off).

5. L supports perl-style embedded string variables for some routines. Example:

     x = "Price of item $item is $thingy.price until Monday"
   
     Is the same as
   
     x = "Price of item " & item & " is " & thingy.price _
         & " until Monday"
   

   Parenthesis are used to clarify boundaries if needed:

     x = "Iliketosquish$(ourstuff)together."
   
     Is the same as
   
     x = "Iliketosquish" & ourstuff & "together."
   

   The VarParse() function can be used to translate embedded variables for those routines that don't automatically support it.

6. Arrays can only contain scalars (strings), not other arrays. It is assumed that a collection/DB engine is used for complex collections instead of pointers to pointers, which are popular with perl. I hate those. (However, one could store the name of other collections, and use Eval() and some other tricks to emulate them.)

7. A statement can be put on the same line in an IF statement using the "do" keyword:

     if x %n> y do foo()
     // is equivalent to
     if x %n> y 
       foo()
     end if
   

   "Do" does not apply to the braces style.

8. A comma in an Out statement simply puts a single space between the arguments.

     out 1, 8    // result: 1 8
   

   This makes debugging a little simpler.

9. Any line that starts with "<" is assumed to be HTML output. (Although L is not exclusively meant to be a web language, it has such web-friendly features.) Another HTML feature is the HTML block:

     HTML
       <table border=1>
       <tr><td> Hey hey hey! </td></tr>
       </table>
     endHTML
   

   Of course, out() would also output as CGI/HTML output in a web context.

10. All declared variables are automatically initialized with a zero-length string.

11. Variable and subroutine names are not case sensitive. Case sensitive tokens do not belong in any scripting language.

12. Block-level declarations are supported.

      if foo()
         var x = 3
         blah()
         var y[]
         blah()
      end if
      // x and y[] are no longer available here
      

13. Globals can be declared inside of any routine, not just the main routine.

14. It is up to the programmer to check for any missing fixed-position parameters. Missing parameters default to an empty string.

15. Number sign can be used to represent the variable on the left side of an assignment.

      foobar = replace(foobar, "mary", "marry")
      foobar = replace(#, "mary", "marry")  // same as above
      x = x + 5
      x = # + 5   // same as prior
      

16. In a for-to loop, to avoid going through the loop at least once, you must specify a step. Example:

      for i = 1 to x step 1
        blah()
      end for
      

    If x is less than 1, then the loop will not be entered. Similarly, in for i = 10 to x step -1, if x is greater than 10, then the loop will not be entered.

17. L has only one type instead of two by making all variables be associative arrays.

      myvar = 5    // is the same as:
      myvar.~sys_lang_value = 5
      

    In such a setup a typical variable assignment would be nothing more than a shortcut for the "value" Hidden Entry. For efficiency purposes, the interpreter can perhaps assume a simple variable until other assignments or references imply an array. However, this is an implementation issue and not a syntax issue.

    This also suggests an approach to have a strong-typing option in the future: there could be a "~sys_lang_type" hidden entry. I don't recommend it at this stage, but it shows how flexible L can be.

18. The original comparison modifier syntax ("if x n> 3") seemed to bother some people. Therefore, I decided that there should be an explicit symbol to separate the variable from the modifier:

      if x %n> 3
        foo
      end if
      

    I am open to suggestions. Perhaps just keep it the way it was, but make the parenthesis mandatory:

      if x (n>) 3
      

    Perhaps curly braces could be used instead:

      if x {n>} 3
      

    For now, I favor the "%" style.

Reserved Words

canEval() - returns true if no syntax errors. 
     Does not check the existence of routines 
     nor variables.
canNum() - Boolean, if an expression can evaluate
     to a number.
case
clone() - Copies an array
cmp()  - Functional version of comparitors
         Example: if cmp(x , "n>", y)
         or:      if cmp(x , "%n>", y)
dump()  - Shows entire array/variable contents. Mostly for
          debugging. Example:
          out dump(x, "/n")  // output follows:
          foo = "Bill Jones"
          gork = 7.2
          ~sys_zork = "blah"
          (etc....)
each - used with for...each constructs
eval()
exec()     - Execute a string as if it was code
elements() - Number of array elements (excluding hidden)
for
has() - Boolean, if array has given element
     Example: has(anArray, "bar")
if
inherit - allows defined routine to inherit 
     parent's scope. Ex: sub x(y) inherit
my  - scope specifier for innermost scope
new - (reserved for possible future use)
numericate() - makes subscripting more "number
     friendly"
otherwise - used with case statements
out - output
outln - output with a linefeed (like C println)
package (reserved for future use)
return
recent() - similar to 'this' but only returns the
     text name of the array
remove() - remove element from array
scope   (for future implementation)
select - define a case statement
set() - set user-defined Hidden Entry
slset() - set language-defined Hidden Entry
static - define variables that do not 
     reinitialize between subroutine calls.
sub - declare a subroutine or function
sys  (reserved for future use)
this - alias to the most recently referenced array
varparse() - embedded string parsing
while


RESERVED HIDDEN ENTRIES

[to be inserted later]

FIST - A meta-language

I call a draft of such a language "FIST". It is kind of based on the idea of LISP, but with the function name on the left of the parenthesis instead of inside. Everything in the language would be a function, including IF, End-IF, and function declarations or statements. This greatly simplifies its syntax structure. However, unlike LISP, the parenthesis are usually not "far reaching" for statement blocks. Blocks are marked by a set of two functions using an X/endX convention instead of parenthesis. Here is an example of some L code translated into FIST:

  //*** L CODE ***
  myFunc(1, amt, "hey", #foo 7)   // call a function
  sub myFunc(a, @b, c)
    print("foo=" & params.foo)
    if a %n> b
      print("total=" & a + b)
    end if
    r = b - a
    return(r)
  end sub

  //*** FIST CODE ***
  myFunc(1, amt, "hey", #foo 7)
  sub("myFunc")
    setParam("1", "a", true)
    setParam("2", "b", false)  // not alterable
    setParam("3", "c", true)
    print(cat("foo=", params["foo"]))
    if(cmp(a, "n>", b))
      print(cat("foo=", add(a,b)))   
    endif()  
    assign(r, subtract(b, a))
    return(r)
  endsub()

Variations

• Use "=" named parameters instead of "#". I prefer the "#" convention, but equal is more established. Note that Fist resembles XML more if we do this. It almost reminds me of the ColdFusion language.

• An equal-sign for assignment can be used if not nested within another function. Thus, one could optionally do this:

       x = cat(a, b)
  

  Instead of:

       assign(x, cat(a, b))
  

  Although this breaks the "pure-function" rule, assignment is perhaps frequent enough to justify complicating syntax for it. It could also make Fist useful as a relational language foundation.