Introduction to the core.ns application framework

Time-tested approach to organizing data, configuration and a code.
core.NS
Namespace-based functional Application Framework

Here is what you will get with core.NS:
1. A clear and time-tested approach to the organizing your application space: “Everything is a file”
2. There is no classes and objects, only filesystem-like environment.
3. Code represented by partially-evaluated functions.
4. Every function in core.NS is not a pure function, as it receives a reference to a full namespace and can
change behavior based on the state of the Namespace.
5. Device-like entities with a local context passed to a partially-evaluated functions.
6. Data resides alongside with code in Namespace.
7. Embedded parser and a help system for command-line arguments.
8. Ability to add functions and data into a Namespace from Python modules.
9. Ability to add data into a Namespace using safe configuration files.
What this approach have to offer ?

TABLE OF
CONTENTS
ABOUT ME
A few words about an author of
core.NS
A FILESYSTEM
in-app Namespace
01
03
02
04
05
06
A FUNCTIONS
Application functionality
DEVICES
Context-sensitive functions
CFG
Configure namespaces
CONCLUSION
What couldn’t you do with
core.NS

A Filesystem
01
When everything is a file. Or at least looks like a file

My name is Vladimir Ulogov, and I’ve been around a while. Being a Python
software developer among other things, I was and still are researching for a better
way to organize a Python application. Better way to bring all components
of the application together.
This interest is not just for “pure research”. I’ve developed a few tools and
Applications using Python programming language over a time. Monitoring systems
and CI/CD instrumentation for configuration and package deployment. A database
for storing graph structures and system authentication and authorization layer.
Custom-built web/rpc server for serving monitoring metrics and statistics.
Data processing pipelines and applications. Security data preprocessing application and other instrumentation, covering
wide needs of IT landscape.
Along with end-user applications, a number of Python modules, was developed by me over time. The most
recent, are covering topics of cryptography, expert system shells, data processing and conversion. Newly found interest
in ML will expand my horizons.
ABOUT ME

A Filesystem
02
When everything is a file. Or at least looks like a file

There are hundreds, if not thousands Application Frameworks for a Python programming language.
Some of them withstand a test of time, some are fairly new. Some of those frameworks are Object-Oriented, some of
them …. well, most of them are based on representation of your application through objects. Application logic are
represented as a cascade of methods calls and recently, with newly-found fancy for the microservices, cascade of the
distributed RPC calls. In the new, distributed world, we are experiencing a strong call for the new Application
Frameworks, which will bring together both local and a distributed logic in a simple, straightforward way. The way, that
will at least looks familiar for may developers and the way that will not bring artificial barriers between data and code,
but rather bringing data and code together, providing clear naming distinction.
There is nothing new in this approach, which unified data and a code in application space. Because same
approach was implemented in Unix decades ago. This approach is having a name: “Everything is a file”. If you are not
familiar with a Unix OS, “everything is a file” means that there is only one way to deal with everything that’s it is in Unix by
treating that’s everything, including application code is a file, that’s you can read, write and use according with your
permissions. Every entity in Unix, well … almost every entity (but every entity in OS Plan9) do have a file name. There are
API interface, defining how you can access those files and there are standards defining where you can find different
elements of the OS. From a binary programs to data, configuration and a log files.
A FILESYSTEM

Of course, an application is having much more simple structure than a full-sized OS. Almost everything in
application space are in memory, so when we are saying, that we are implementing the same approach in core.NS as it
is in Unix OS, we do not have to actually implement an in-application filesystem, which are real, “on the disk” filesystem.
In the core of our Application Framework is a “virtual FS”. A way to organize our application, when each and every entity
having a unique file path-like name. Can be addressed by this unique name and permitted operations can be performed
with this entity, addressable by it’s name. So, our approach is very similar to Unix OS approach, but there are
differences.
In core.NS - everything is a data, addressable by it’s path in in-memory namespace. This in-memory
namespace, provides a references to a locally placed data, functions and both local and network resources. There are
very important point there: current core.NS namespace implementation are in-memory structure. The rationale for that is
a speed. In the future, I will provide on-disk namespace extension.
To make core.NS namespace familiar to a crowd, I’ve took a liberty to organize default namespace in the
same way as Unix OS file system organisation. There are /bin, /sbin, /home, /etc, /usr/local/bin and the other
placeholders, that you are familiar with.
A FILESYSTEM

So, if you want to assign a variable with a value in the core.NS, you have to specify the path for that
variable. Remember, this is not the real path on your application’s host OS filesystem. This is a path in your application’s
virtual FS or we call it a Namespace. There is no local or global variables in a Namespace. Or I rather say: “Everything
stored within a Namespace is a global and available to all, who is having a reference to that Namespace. There are two
types of the elements in core.NS Namespace - folder and data.
1. Folder is a dictionary data, which can contain the references to another folder or data elements. Folder is
referenced by globally-recognized name. Example: /home .
2. Data - finite and atomic element, storing the data and referenced by the globally-recognized name. Example:
/home/Answer .
You can create another data element in /home, for example /home/AnotherAnswer, but you can not
create nether folder, nor data elements under data element, such as /home/Answer/maybeanswer.
A FILESYSTEM

By default, all data elements are read/write. This mean, that you can change them. Both, creating new
data elements with assigned value and change the value of existing data elements are permitted. You can change this
behavior by setting value to a variable /config/var.redefine . If you set the value of this variable to False, all existing
variables will become a read-only. This mean, that you can not change the value of an existing variables. You can create
a new elements and assign the data to those elements, but can not change the value of existing variables.
A FILESYSTEM

How we can start to explore our Namespace ? Let me bring a very simple example:
1. First line of the code will import all high-level functions of core.NS.
2. Second line, will create a new Namespace and return three elements required for working with this
Namespace
a. ns - is a reference to a dictionary. Yes, namespace is nothing but Python dictionary and you can
access it and explore it using standard Python methods.
b. f - partially evaluated function (more about this in the next chapter) for locating and returning
reference of the partially evaluated function stored in the Namespace. Path to this function is
passed as a first (but actually, the second) parameter.
c. F - partially evaluated function for copying the references from Namespace to Python builtins.
3. To explore Namespace, you will need a function V, referenced as /bin/V
4. Stored in local variable V is a partially evaluated function V. It’ll return the value of the data element
referenced by path passed as first (well, second, actually) parameter.
A FILESYSTEM

A FUNCTIONS
03
They are first-class citizens, or they not a functions.

Functions are the first-class citizens in core.NS. This means, that they support all the operations,
available to other elements of the Namespace. You can create data elements containing references to a functions as
you are creating any other data elements. You can reference functional data elements, as you are referencing anything
else on core.NS. So, functional fact number 1, functions are the data elements in Namespace and each function can be
referenced just like any other data element, by there path. For example: /bin/id .
But it is a bit more about a functions then meet an eye. There is no objects in core.NS and while the
functions in framework are not pure, each of them having a reference to a full Namespace as a first parameter.
Functional fact number 2 - you can not take just any Python function and turn it into core.NS function, because to each
function in the framework, we will pass that reference. But when ? During execution ? Would it be too mundane to pass
the same thing multiple times ?
A FUNCTIONS
The answer to the problem is that in Namespace, we are storing a reference to a partially evaluated
function and every time that we are referencing that function, we are receiving a reference not to the
function itself, but rather to a partially evaluated function, exactly as we store it.

What is the “partially evaluated function” ? First, never mix “partial evaluation” or “partial application” to a
currying. Currying and partial evaluation while being related, are two different concepts. This said, “partial evaluation” is a
process of binding a function with one or more parameters. This process returns the reference on this partially evaluated
function, which are may or may not be of the smaller arity, than the function which we will be calling. So, in order to
complete the function arity, we will provide a necessary arguments during the actual function call. Example:
A FUNCTIONS
f ( ns, X, Y, Z) -> N -- Here, we’ve defined a function, that will have a first argument - the reference to a namespace and
three other arguments, named X, Y, Z. The arity of this function is four.
fP := f(ns) -> N -- Here we bind the function with first argument, that is a reference to a namespace. Then we are storing
the reference to this partially evaluated function to the reference fP.
fP(X,Y,Z) -> N -- and here, we are using the reference to a partially evaluated function to call it with the rest of the
arguments. Now our arity is matched to what’s we’ve had to an original function and the process of function evaluation
will be complete.

Let’s take a look at some operations with partially evaluated functions, stored in core.NS Namespace. I
will not go in details of device implementation, this will be a topic for the next chapter. Although, in the line 3, we are
locating partially evaluated function “open”, referenced by path /dev/queue/open and calling it with first (oh well, actually
third) parameter “test”. This function, when called, creates a new “directory” in the Namespace called /dev/q/test, all
required context to the queue and referencing two partially evaluated functions with two parameters already bound.
First, is a reference to a Namespace, second - reference to queue. On line 4 and 5, we can see and example of use newly
created functions. You do not have to store the reference to a function, you can evaluate partially applied function
immediately after referencing. And functions are first-class citizens. You can get an access to a function, using function
for access the data. On line 7, you can see that you don’t have to specify full path to your functions, if they are “in the
PATH”.
A FUNCTIONS

DEVICES
04
Very context-oriented functions

In the previous chapter, I already bought an example of the use of core.NS “device”, without explaining of
what is this and what is this for. In a way, core.NS “devices” are similar to a Unix OS devices. Like in Unix OS, our
devices characterized by the path, normally in /dev Namespace, data elements representing context and a one or a few
partially evaluated functions whose parameters includes a references to a Namespace and a context, specific to that
device.
DEVICES
In this example, you can see, that every device function, have two
references. First parameter is a reference to a global namespace.
Second parameter is a reference to a local namespace, which could
be a reference to a “directory” in which this function is registered, or
to a specific data element, representing relevant context for this
function.

So, let’s us reiterate an example of the device, implementing a queue functionality. Before you start to
work with queue, you have to open it. To do that, you are calling a function of the “global queue driver” and pass as first
(well, actually second) parameter the name of the queue. “Global queue driver” creates directory under “/dev/q” named
after queue name. In our example, that would be a “/dev/q/test”
DEVICES
While “/dev/queue/open” initialize a named queue in “/dev/q”, it
creates a two partially evaluated functions, named “put” and “get”
within /dev/q/test”. The first parameter of each of those function is a
reference to a global Namespace. No surprises here. But the second
parameter is a reference to the specific instance of the queue,
making those two functions “context aware”. As you see, after driver
is initiated, there is no possibility for the error, due to improper
references to a queue. This reference is fixed as a part of the process
of parameters binding.

And if we take a look at the code, accessing to the driver functions, you can see, that not only all details
of context initialization are hidden from you, but also all details of working with this context is also hidden. All you see is
a functions. You are create and use a lightweight functions and not a maze of the objects. And for “end-user”, there is no
difference if you are calling a context-centric and non-contexted function. All those details are hidden, but not too deep.
DEVICES

CFG
05
We just love our config files

There is no “well regulated” Application Framework, without own configuration system. Format of the
configuration files has been inherited from the language called “the Bund” and in fact are the subset of the language. It
been done that way for several reasons:
1. Expressiveness (yes, I found Bund language to be very expressive).
2. Safety. Bund virtual machine not calling nether eval nor apply and parse data elements without calling Python
directly.
3. Tight integration with Namespaces
4. Automatic conversion between Bund and Python and support of all, most practical data types, such as:
1. Integers and Floating point numbers;
2. Strings
3. Boolean
4. Lists
5. Dictionaries
CFG

CFG
First, what you see in this example, is the comments.
Comments are the C-styled, placed between “/*” and “*/”. It is a
pretty standard and already may be familiar to some of you.
Setting variables to a Namespaces are happens within
Namespace definitions, which begins with ”[“ Namespace path “>”
and ends with “;;”. All variable definitions between those statements,
will be translated to a variable in a Namespace. There is no particular
order for a Namespace definition and you can a definitions of the
same Namespaces as many times, as it does make sense to you.
You can also include Namespace definition inside another
Namespace definition.

CFG
Data elements could be defined using direct and
reverse assignments. Regardless, which one you are using, each
definition of the Data element consists of Name and the Value
Direct definition is looks like this: Name [“<-” | “is” ] Value.
Reverse definition shall not cause any stress, after you’ve learned
a direct definition: Value [“->”| “as” ] Name.
You can mix’n’match different definition styles, according to your
preferences.

CFG
You can import configuration from the external resources by its name. But before loading configuration from such
resources, you have to define the list of the “root filesystems” from which you will load your configurations. Here the list of the types of
“root filesystem”:
1. “ftp://hostname/path_to_ftp_directory” - config file will be loaded from FTP server
2. “mem://” - config file will be loaded from in-memory “file system”
3. “osfs://path_to_the_local_directory” - config file will be loaded from the local directory
4. “http://hostname/path_to_http_directory” - config file will be loaded from HTTP server
You shall store those “root filesystms” as list into a data element “/config/cfg.path”, for example as
V(“/config/cfg.path”, [“osfs://.”, “http://www.example.com/configs”])

CFG
There are two functions, available to a user for loading config files:
1. f(“/bin/C”)(configuration file statements) - load config file statements from parameter. Example:
f(“/bin/C”)(V(“/config/data/main.cfg”)), this function call will read config file statements stored in data element located in
“/config/data/main.cfg” and load those statements.
2. f(“/bin/Cfg”)(config file name) - locate file “test.cfg” file within the list of the “root filesystems”, stored in “/config/cfg.path”

ZEN OF CORE.NS
06
Conclusions and afterwords

➔ Open access with clearly defined data, functions and access logic are better than the maze of
objects, globals and procedures.
➔ A well regulated Namespace, being necessary to the Application architecture, the right of the
developer to store data and code inside an application as he or she deems appropriate, shall not
be infringed.
➔ There is nothing wrong with external configuration files, defining values in your Namespace.
➔ Variables in your Namespace have a right to be read-only.
➔ Partial evaluation are better than carrying repetitive parameters.
➔ Don’t be afraid to create a new partially applied functions with clearly defined parameters.
➔ Functions are first-class citizens, or bust !
➔ Objects are not necessary evil, but sometimes, they are not necessary.
➔ Non-pure functions with a reference to a Namespace are fine too.
➔ Unix filesystem layouts are beautiful and time-tested.
➔ Context-sensitive devices are the way to organize your low level logic and hide unnecessary
details.
ZEN OF CORE.NS

Fin
Now, you’ve learned enough about core.NS, to get you going. Give it a try,
create your applications and unleash the power of this framework.
You can get in touch with me, via LinkedIn:
https://www.linkedin.com/in/vladimirulogov/
You can explore full source code of core.NS on GitHub.
https://github.com/vulogov/core.ns
Any bug reports are appreciated

Introduction to the core.ns application framework

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Introduction to the core.ns application framework

Similar to Introduction to the core.ns application framework (20)

More from Vladimir Ulogov

More from Vladimir Ulogov (9)

Recently uploaded

Recently uploaded (20)

Introduction to the core.ns application framework