Expression Parser Release [2x0]

Introduction
Welcome to the release of Expression parser version 2x0.
This is going to be a short thread as all the necessary information can be found in my other threads created about ep, they can be seen at the bottom of this page.

What is Expression Parser?
Ep, which by the way is short for Expression Parser, is a new scripting language created for the sole purpose of being used by MPGH members exclusively. It's meant to work along the side of C++, meaning you can directly use ep in your C++ Applications by simply including the ep headers and libraries. The now improved C++ portability means you can have ep arguments (variable_list) automatically deduced into arguments at C++ compile time.

I wanna learn more, how does it work?
Expression Parser Theoretical Introduction
Expression Parser Practical and Syntactical Introduction

Changelog
[...]

Version 1x8

Code:

- Added more C++ type safety.'
  You can no longer create value_reference of int and cast it to e.g double.
  
- Fixed an assignment bug (I forgot to change a function)

- Remade error message system completely and it now has better error messages.

- Made some changes to value reference.

Version 1x9

Code:

- Added 'type_info' type (use either typeid() or type_info() to construct)
  it provides the basic utilites needed to check the run-time type information
  of variables etc.
  
 - Added standard type nullptr_t (use nullptr keyword/variable)
  
- Added ability to import dll's that are using the EP plugin format
  (instead of specifying it inside of the ep::module constructor, it
   can still be done like that though)
  
- Added optional new import syntax, totally didn't steal it from go ;)
  You can now import multiple libraries or files with one import statement.
  Example:
	import( bo, 
	        ios, 
			vector,
			"somefile.ep",
			"somelibrary.dll" );
  
- Optimized some of the evaluation code (Move semantics etc)

- Fixed an internal parser error when converting infix to (not actual) postfix.
  The following expression would fail to be parsed because a trailing '(' was
  added to the end of the postfix expression (no parentheses allowed):
	typeid( "hi" )->bare_equal( typeid( 6 ) )
  This is now fixed, and should produce a valid postfix output.

- Fixed a bug when accessing member methods, it didn't check the arity
  of the arguments on the stack, so it caused an uncatchable exception.
  
- Fixed a few ep_error bugs (lol)

- Fixed a bug in 'vector::for_each' using the wrong argument.

- Fixed a value_reference bug (it would attempt to destroy references)

- Fixed a bug in bo::operator! (doing it with any other type than 'floating' would throw a conversion error)

- Fixed yet another access bug.
  The error would happen whenever you did something like this:
	type->get()->method( argument );
  The postfix output would be falsely generated here, it would attempt to access 'method'
  from 'argument', instead of the rvalue returned from 'type->get()'.

Version 2x0

Code:

- Improved *.oep parsing.

- Modules are now placed in %temp_dir%/eptemp_*.dll
  A cleaner for these temp files will be created at a later time.
  Use the cleanup tool found here to clean them up once in a while.

- Improved tokenizer.
  It will now discard unknown sequences, instead of throwing an error
  note that, this may potentially cause undefined behaviour at times.

- Added optional export function "get_prerequisites", you may specify a one lined
  import statement in this method, that is if your plugin requires a prerequisite
  such as the standard "vector" library.
  
- Added 'decltype' method, it searches the type systme and finds a specified type.
  example:
	decltype( "integral" ); returns the type_info group for "integral".
 
- Added 'exit' function that stops executing the current script.
  Note that, using this function will not exit the current appliaction,
  it only throws an exception catchable by evaluater, and stops evaluating
  from there on, triggering all the cleanup methods and such.

- Added the 'specialize' keyword, it can be used before a method, it is a
  way of doing runtime 'overloads' (if you will) of methods, you can have
  a logical expression in it's scope (defined by the operators < and >).
  A simple example would be a method to see if 'a' is of type 'integral':
  
	// this is the default method that is called if all 'overloads' fail.
	method is_integral( &a /* The argument */ )
	{
		return 0; // hence we return false.
	};
	
	// Note that the syntax of a specializer is as follows:
	// specialize + ['<'] ([expression] + [';']) + ['>']
	specialize<typeid( a ) == decltype( "integral" );>
	method is_integral( &a )
	{
		// This method will only ever be called if the specializers 
		// expression evalutes to true.
		return 1; // And thus; we know argument 'a' is of type 'integral' 
	};

Sincerely,
Yamiez.

expression parser_mpgh.net.rar

You said it's an interpreted language, but then you wrote this part which tells me it's compiled:

Ep has a few ways to optimize the interpreted code, you see having no optimization would result in poor performance, as humans often like to follow guidelines that prefer elegancy over performance. The optimization algorithms work to reformat the provided code blocks into a more, performance oriented block of code. The core optimization algorithms consist of three sub-algorithms that work independently with said sub-algorithms.

Return omit optimization is the very first algorithm being ran on the provided block of code, this doesn’t really increase performance in any, all this does is reduces the memory usage of the block of code by omitting all statements after a return statement, however only in the current scope of said return statement. (Note: return omit is a modification of dead code optimization)

Dead code omit optimization is specifically running second, because firstly; this optimization could possibly add extra overhead to the optimizer if it was running before the return omit algorithm, considering that empty expressions may be present after a return statement. Secondly, the third algorithm needs to run after this one, because otherwise the third algorithm may go through and change these empty, useless, and overhead creating statements. The name defines the algorithm very well; it simply omits all statements that have nothing but an empty expression that doesn’t modify anything.

Constant folding optimization is a little more complex to implement than the previously mentioned ones, but it’s purpose is to reduce the amount of evaluations needed to be done at run-time, thus increasing the performance and at the same time reducing the code size. Because having the interpreter evaluate 10 to the power of 9 at run-time isn’t exactly performance friendly compared to directly having the parser replace the equation of 10 to the power of 9 with a constant number equalling a billion.
I hope all of this makes sense to the less experienced developers reading this, I tried the best I could with my limited experience in this subject to explain these slightly complex algorithms used to optimize code blocks.

if it actually is, why should you create DLLs?

Code:

void main()
{
     auto module = std::make_shared<ep::module>( );
     try 
     {
          module->eval_file( "script.ep" );    
     }
// ...
     auto function_object = module->get_method_object<void( const std::string& )>( "ep_method" );
     // Call it right here in C++
     function_object( "Call from a function object." );
    // Call it directly with the module
    module->call_method<void>( "ep_method", "Call directly from module ptr" );

you show this code too, which actually tells me it's compiled -> it seems this dll is the compiled C++ generated file which imports the .ep file with eval_file. There are some contradictions, but you also give just a few informations about it.

First of all, I know the documentation is very messy as of this moment, things are clustered together with things that aren't even valid anymore. I should have more time now to clean things up as I have holidays, but then again, I also have plans. School takes up a lot of my time, especially since I travel 4+ hours a day to and from school.

Originally Posted by javalover

You said it's an interpreted language, but then you wrote this part which tells me it's compiled:

I don't see how that part can tell you it's compiled, an interpreted language can have just the same optimizations as a compiled language. Usually when we say, compiled languages, we're talking about code that gets compiled into straight up machine code. Ep scripts are never compiled into machine code, but into seperate blocks (called "code blocks"), each block representing some kind of token as to what the action is. For example, is_block_type meaing, it is some type of statement that may lead into it's own code block (such as an if statement). There are many different ones, and the interpreter loops through these blocks many times to make modifications that can only be seen after an entire block has been "compiled" (note that, I use the word compiled, in a non-binary way, I am simply talking about removing certain flags, and data from statements).

After the aforementioned process, the interpreter well, interprets these blocks (seen in module::eval_internal, statement::eval_or_eval_and_return, and expression::evaluate_actions).

Originally Posted by javalover

if it actually is, why should you create DLLs?

I haven't been very clear as to why this is, this will only happen when you use the protected ep (*.oep or obfuscated expression parser code, or short oep) format. And as of right now, the "compiler" (It never compiles into machine code!) hasn't been released yet, or atleast newer version. It creates these temporaries because the oep formats ability to store .dll's and ep script code. So when naturally, in ep you can import dll's using the import keyword, doing something like

Code:

import "mydll.dll";

With this information, the compiler will:
First, check syntax.
Secondly, resolve the imports. (Which will remove the import code block entirely, and instead just store a temporary string for later.)
Thirdly, generate the code. (Which will look in the relative path for all of the modules found, and just mash them into the oep file with the correct oep format.)
Lastly, save the code.

A real world example:

So running the "compiler" on the "ep.ep" script (which imports first, second, third, and fourth dll's) will generate an output file of approximately the size of all these files.

And now we get to the part where the temporary files are created.

Code:

ep::statement_list ep::module::deobfuscate_contents( std::string & contents )
{ 
	if ( obfuscation_algorithm( contents ) )
	{
		auto header = *reinterpret_cast< const _Header* >( contents.data( ) );
		auto copy_contents = contents;
		copy_contents.erase( 0, header.size );
		contents.erase( header.size, contents.size( ) );
		contents.erase( 0, sizeof( _Header ) );

		auto copy = copy_contents;
		std::vector<std::string> libs;
		while ( obfuscation_algorithm( copy ) )
		{
			header = *reinterpret_cast< const _Header* >( copy.data( ) );
			copy_contents = copy.substr( 0, header.size );
			copy_contents.erase( 0, sizeof( _Header ) );
			copy.erase( 0, header.size );
			libs.emplace_back( std::move( copy_contents ) );
		}
		copy.erase( 0, sizeof( _Header ) );
		libs.emplace_back( std::move( copy ) );

		statement_list statements;
		ep::compiled_plugin_loader loader( std::move( libs ) );
		for ( auto &x : loader.get_plugins( ) )
		{
			auto load = x->get_load_fn( ); 
			auto name = x->get_name_fn( );
			auto preq = x->get_prerequisites_fn( );
			if ( !load || !name )
				throw std::exception( "unable to load *.oep file because; one of it's modules are invalid." );
			std::string sname( name( ) );
			if ( is_included( sname ) )
				continue;
			includes_.emplace_back( sname );
			if ( preq )
			{
				cpputils::cpp_tokenizer tokenizer;
				tokenizer.tokenize( preq( ) );
				if ( tokenizer.begin( )->get_block( ) == "import" )
					parse_import( tokenizer.begin( ) + 1, tokenizer.end( ), statements );
			}
			load( engine_ );
			libs_.emplace_back( x->get_path( ) );
		}
		return statements;
	}
	else
		contents.erase( 0, sizeof( _Header ) );
	return{ };
}

I'm not going to explain this code much more than the comments.

Now you see, libraries in ep are nothing more but a simple plugin, here's an example of the threading library:

Code:

extern "C" __declspec( dllexport ) void load_plugin( ep::dispatch_engine_ptr engine )
{
	ep::add_type( engine,
				  "thread",
				  [engine]( )
	{
		return std::make_shared<ep::value_reference>( new thread_proxy( ),
													  engine->get_type_info( "thread" ) );
	},
				  []( ep::value_reference *ptr )
	{
		delete ptr->get_pointer<void>( );
	},
		[]( ep::value_reference *ptr )
	{
		return std::make_shared<ep::value_reference>( new thread_proxy( *ptr->get_pointer<thread_proxy>( ) ),
													  ptr->type_info( ) );
	},
		"start",
		[engine]( thread_proxy *obj, const std::string &func )
	{
		return std::make_shared<ep::value_reference>( obj->start( func, engine ),
													  engine->get_type_info( "thread" ),
													  false );
	},
		"is_running",
		[engine]( thread_proxy *obj )
	{
		return std::make_shared<ep::value_reference>( new int( obj->is_running( ) ),
													  engine->get_type_info( "integral" ) );
	},
		"wait",
		[engine]( thread_proxy *obj, std::chrono::milliseconds &milli )
	{
		obj->wait( milli );
		return ep::void_return;
	},
		"suspend",
		[]( thread_proxy *obj )
	{
		obj->suspend( );
		return ep::void_return;
	},
		"resume",
		[]( thread_proxy *obj )
	{
		obj->resume( );
		return ep::void_return;
	},
		"kill",
		[]( thread_proxy *obj )
	{
		obj->kill( );
		return ep::void_return;
	},
		"sleep",
		[]( thread_proxy *obj, std::chrono::milliseconds &milli )
	{
		obj->sleep( milli );
		return ep::void_return;
	} );

	ep::add_type( engine,
				  "milliseconds",
				  [engine]( const int &x )
	{
		return std::make_shared<ep::value_reference>( new std::chrono::milliseconds( x ),
													  engine->get_type_info( "milliseconds" ) );
	},
				  []( ep::value_reference *ptr )
	{
		delete ptr->get_pointer<void>( );
	},
		[engine]( ep::value_reference *ptr )
	{
		return std::make_shared<ep::value_reference>( new std::chrono::milliseconds( *ptr->get_pointer<std::chrono::milliseconds>( ) ),
													  ptr->type_info( ) );
	},
		"count",
		[engine]( std::chrono::milliseconds *ptr )
	{
		return std::make_shared<ep::value_reference>( new int( ptr->count( ) ),
													  engine->get_type_info( "integral" ) );
	} );

	engine->add_method( ep::make_method( "current_thread",
										 [engine]( )
	{
		return std::make_shared<ep::value_reference>( new thread_proxy( _Current_thread( ) ),
													  engine->get_type_info( "thread" ) );
	} ) );
}

These plugins are required to use C++ extensions in ep, because technically, ep has no idea about C++, internally that is.

However, C++ has an idea about ep. And as such can call ep methods directly in C++, this isn't compiling it, it's just a kind of smart way to do it.

How ep function objects are created:

Code:

lambda_ = [this]( variable_list arguments, dispatch_engine_ptr engine )
		{
			auto state = engine->create_new_dispatch_state( get_name( ), get_arguments( ), std::move( arguments ), engine->get_global_state( ) );
			for ( auto &x : get_statements( ) )
				if ( x->eval_or_eval_and_return( engine, state, engine->get_debugger( ) ) )
					return std::move( x->get_return( ) );
			return ep::void_return;
		};

Not very hard!

And to clarify, C++ and ep communicates a lot at compile time, I utilize callable traits to get the type of each argument in C++ callables, to be able to generate the specified ep wrapper code.

head aches

C++ to EP

Code:

namespace ep
{
	namespace details
	{

		namespace detail
		{
			template<unsigned... digits>
			struct to_chars
			{
				static const char value[];
			};

			template<unsigned... digits>
			const char to_chars<digits...>::value[] = { 'a', 'r', 'g', ( '0' + digits )..., 0 };

			template<unsigned rem, unsigned... digits>
			struct explode : explode<rem / 10, rem % 10, digits...>
			{ };

			template<unsigned... digits>
			struct explode<0, digits...> : to_chars<digits...>
			{ };
		}

		template<unsigned num>
		struct arg_pos_to_string : detail::explode<num>
		{ };

		struct _Ref_ptr
		{ };

		template<typename T>
		struct callable_traits;

		template<typename Ret, typename...Params>
		struct callable_traits<Ret( Params... )>
		{
			static constexpr auto size = sizeof...( Params );
			using result_type = Ret;
			typedef Ret( *signature_type )( Params... );

			template<size_t pos>
			struct arg_t
			{
				using type = typename std::tuple_element<pos, std::tuple<Params...>>::type;
			};
		};

		template<typename _Deduce, bool ref = std::is_base_of<_Ref_ptr, _Deduce>::value>
		struct _Var_deducer
		{
			static _Deduce deduce( ep::variable_ptr &ptr )
			{
				return *ptr->get_ref( )->clone( )->get_pointer<_Deduce>( );
			}
		};

		template<typename _Deduce>
		struct _Var_deducer<_Deduce*, false>
		{
			static _Deduce *deduce( ep::variable_ptr &ptr )
			{
				return ptr->get_ref( )->get_pointer<_Deduce>( );
			}
		};

		template<typename _Deduce>
		struct _Var_deducer<_Deduce&, false>
		{
			static _Deduce &deduce( ep::variable_ptr &ptr )
			{
				return *ptr->get_ref( )->get_pointer<_Deduce>( );
			}
		};

		template<typename _Deduce>
		struct _Var_deducer<_Deduce, true>
		{
			static _Deduce deduce( ep::variable_ptr &ptr )
			{
				return _Deduce( ptr );
			}
		};

		template<>
		struct _Var_deducer<variable_ptr, false>
		{
			static variable_ptr deduce( ep::variable_ptr &ptr )
			{
				return ptr;
			}
		};

		template<size_t pos, size_t max, typename Traits, typename Ret = typename Traits::result_type>
		struct _Caller
		{
			template<typename Func, typename...Deduced>
			static auto
				_Process_and_call( ep::variable_list &a,
								   Func &&f,
								   Deduced&&...deduced )
			{
				using arg_type = typename Traits::template arg_t<pos>::type;
				return _Caller<pos + 1, max, Traits>::_Process_and_call( a,
																		 std::forward<Func>( f ),
																		 std::forward<Deduced>( deduced )...,
																		 _Var_deducer<arg_type>::deduce( a[pos] ) );
			}
		};

		template<size_t max, typename Traits, typename Ret>
		struct _Caller<max, max, Traits, Ret>
		{
			template<typename Func, typename...Deduced>
			static typename Traits::result_type
				_Process_and_call( ep::variable_list &a,
								   Func &&f,
								   Deduced&&...deduced )
			{
				return std::forward<Func>( f )( std::forward<Deduced>( deduced )... );
			}
		};

		template<size_t max, typename Traits>
		struct _Caller<max, max, Traits, void>
		{
			template<typename Func, typename...Deduced>
			static ep::value_reference_ptr
				_Process_and_call( ep::variable_list &a,
								   Func &&f,
								   Deduced&&...deduced )
			{
				std::forward<Func>( f )( std::forward<Deduced>( deduced )... );
				return ep::void_return;
			}
		};

		template<size_t pos, size_t max, typename traits>
		struct _Arg_emplacer
		{
			static void emplace( ep::method_argument_list &list )
			{
				using arg_t = typename traits::template arg_t<pos>::type;
				list.emplace_back( new ep::method_argument( arg_pos_to_string<pos>::value,
															std::is_reference<arg_t>::value || 
															std::is_pointer<arg_t>::value || 
															std::is_base_of<_Ref_ptr, arg_t>::value ) );
				_Arg_emplacer<pos + 1, max, traits>::emplace( list );
			}
		};

		template<size_t max, typename traits>
		struct _Arg_emplacer<max, max, traits>
		{
			static void emplace( ep::method_argument_list &list )
			{ }
		};

		template<typename T>
		struct _Build_method_from_t
		{
			template<typename _Invalid>
			static ep::method_ptr _Build( std::string, _Invalid &&invalid )
			{
				throw;
			}
		};

		template<typename Ret, typename T, typename...Params>
		struct _Build_method_from_t<Ret( T::* )( Params... ) const>
		{
			template<typename _Func>
			static ep::method_ptr _Build( std::string name, _Func &&functor )
			{
				using traits = callable_traits<Ret( Params... )>;
				ep::method_argument_list args;
				auto method = [func = std::function<Ret( Params... )>( std::forward<_Func>( functor ) )]( ep::variable_list fargs,
																										  ep::dispatch_engine_ptr )
				{
					return _Caller<0, traits::size, traits>::_Process_and_call( fargs,
																				func );
				};
				_Arg_emplacer<0, traits::size, traits>::emplace( args );
				return std::make_shared<ep::method>( std::move( name ),
													 std::move( args ),
													 std::move( method ) );
			}
		};

		template<typename Ret, typename T>
		struct _Build_method_from_t<Ret( T::* )( ) const>
		{
			template<typename _Func>
			static ep::method_ptr _Build( std::string name, _Func &&functor )
			{
				using traits = callable_traits<Ret( )>;
				ep::method_argument_list args;
				auto method = [func = std::function<Ret( )>( std::forward<_Func>( functor ) )]( ep::variable_list fargs,
																								ep::dispatch_engine_ptr )
				{
					return func( );
				};
				_Arg_emplacer<0, traits::size, traits>::emplace( args );
				return std::make_shared<ep::method>( std::move( name ),
													 std::move( args ),
													 std::move( method ) );
			}
		};

		template<typename T>
		struct _Build_member_method_from_t
		{
			template<typename _Invalid>
			static ep::method_ptr _Build( std::string name,
										  _Invalid && )
			{
				static_assert( false, "_Build_member_method_from_t expects callable object as T" );
			}
		};

		template<typename Ret, typename T, typename...Params>
		struct _Build_member_method_from_t<Ret( T::* )( Params... ) const>
		{
			template<typename _Func>
			static ep::method_ptr _Build( std::string name,
										  _Func &&functor )
			{
				using traits = callable_traits<Ret( Params... )>;
				ep::method_argument_list args;
				auto method = [func = std::function<Ret( Params... )>( std::forward<_Func>( functor ) )]( ep::variable_list fargs,
																										  ep::dispatch_engine_ptr )
				{
					return _Caller<0, traits::size, traits>::_Process_and_call( fargs,
																				func );
				};
				_Arg_emplacer<1, traits::size, traits>::emplace( args );
				return std::make_shared<ep::method>( std::move( name ),
													 std::move( args ),
													 std::move( method ) );
			}
		};

		template<typename Ret, typename T>
		struct _Build_member_method_from_t<Ret( T::* )( ) const>
		{
			template<typename _Func>
			static ep::method_ptr _Build( std::string name,
										  _Func &&functor )
			{
				using traits = callable_traits<Ret( )>;
				ep::method_argument_list args;
				auto method = [func = std::function<Ret( )>( std::forward<_Func>( functor ) )]( ep::variable_list fargs,
																								ep::dispatch_engine_ptr )
				{
					return _Caller<0, traits::size, traits>::_Process_and_call( fargs,
																				func );
				};
				_Arg_emplacer<1, traits::size, traits>::emplace( args );
				return std::make_shared<ep::method>( std::move( name ),
													 std::move( args ),
													 std::move( method ) );
			}
		};

		template<typename..._Specialization>
		struct create_type_impl;

		template<>
		struct create_type_impl<>
		{
			static ep::type_info_ptr _Do( ep::type_info_ptr ti )
			{
				return std::move( ti );
			}
		};

		template<typename Str, typename T, typename..._Specialization>
		struct create_type_impl<Str, T, _Specialization...>
		{
			static ep::type_info_ptr _Do( ep::type_info_ptr ti,
										  Str &&str,
										  T &&t,
										  _Specialization&&...spec )
			{
				ti->add_method( std::move( _Build_member_method_from_t<decltype( &T::operator() )>::_Build( std::forward<Str>( str ),
																											std::forward<T>( t ) ) ) );
				return create_type_impl<_Specialization...>::_Do( std::move( ti ),
																  std::forward<_Specialization>( spec )... );
			}
		};

		template<typename Str, typename T>
		struct create_type_impl<Str, T>
		{
			static ep::type_info_ptr _Do( ep::type_info_ptr ti,
										  Str &&str,
										  T &&t )
			{
				ti->add_method( std::move( _Build_member_method_from_t<decltype( &T::operator() )>::_Build( std::forward<Str>( str ),
																											std::forward<T>( t ) ) ) );
				return std::move( ti );
			}
		};

		template<size_t arity, size_t mod = arity % 2>
		struct _Check_arity_impl;

		template<typename..._Arity>
		struct _Check_arity
		{
			static constexpr bool value = _Check_arity_impl<sizeof...( _Arity )>::value;
		};

		template<>
		struct _Check_arity<>
		{
			static constexpr bool value = true;
		};


		template<size_t arity>
		struct _Check_arity_impl<arity, 0>
		{
			static constexpr bool value = false;
		};

		template<size_t arity, size_t mod>
		struct _Check_arity_impl
		{
			static constexpr bool value = true;
		};
	}

	template<typename _Dtor, typename _Ctor, typename..._Specialization>
	static ep::type_info_ptr create_type( std::string name, _Dtor &&dtor, _Ctor &&ctor, _Specialization&&...spec )
	{
		static_assert( !details::_Check_arity<_Specialization...>::value,
					   "uneven arity, expects arity of pattern (StrConvertable, Callable, Continuation...)" );
		return details::create_type_impl<_Specialization...>::_Do( std::make_shared<ep::type_info>( std::move( name ),
																									std::forward<_Dtor>( dtor ),
																									std::forward<_Ctor>( ctor ) ),
																   std::forward<_Specialization>( spec )... );
	}

	template<typename _Ctor, typename _Dtor, typename _Clone, typename..._Specialization>
	static void add_type( ep::module_ptr &module,
						  std::string name,
						  _Ctor &&ctor,
						  _Dtor &&dtor,
						  _Clone &&clone,
						  _Specialization&&...spec )
	{
		auto engine = module->get_engine( );
		auto type = create_type( name,
								 std::forward<_Dtor>( dtor ),
								 std::forward<_Clone>( clone ),
								 std::forward<_Specialization>( spec )... );
		engine->add_method( details::_Build_method_from_t<decltype( &_Ctor::operator() )>::_Build( std::move( name ),
																								   std::forward<_Ctor>( ctor ) ) );
		engine->add_type( std::move( type ) );

	}

	template<typename _Ctor, typename _Dtor, typename _Clone, typename..._Specialization>
	static void add_type( ep::dispatch_engine_ptr &engine,
						  std::string name,
						  _Ctor &&ctor,
						  _Dtor &&dtor,
						  _Clone &&clone,
						  _Specialization&&...spec )
	{
		auto type = create_type( name,
								 std::forward<_Dtor>( dtor ),
								 std::forward<_Clone>( clone ),
								 std::forward<_Specialization>( spec )... );

		engine->add_method( details::_Build_method_from_t<decltype( &_Ctor::operator() )>::_Build( std::move( name ),
																								   std::forward<_Ctor>( ctor ) ) );
		engine->add_type( std::move( type ) );
	}

	template<typename Str, typename T>
	static auto make_method( Str && str,
							 T &&t )
	{
		return details::_Build_method_from_t<decltype( &T::operator() )>::_Build( std::forward<Str>( str ),
																				  std::forward<T>( t ) );
	}

	template<typename T, typename...Ts>
	static plugin_list load_plugins_impl( plugin_list &output, T &&t, Ts&&...ts )
	{
		output.emplace_back( load_plugin( std::forward<T>( t ) ) );
		return load_plugins_impl( output, std::forward<Ts>( ts )... );
	}

	template<typename T>
	static plugin_list load_plugins_impl( plugin_list &output, T &&t )
	{
		output.emplace_back( load_plugin( std::forward<T>( t ) ) );
		return output;
	}

	template<typename...Ts>
	static plugin_list load_plugins( Ts&&...ts )
	{
		plugin_list list;
		return load_plugins_impl( list, std::forward<Ts>( ts )... );
	}

	// wrapper for passing variable_ptr as reference
	struct argument_ptr_ref
		: public details::_Ref_ptr
	{
		using ref = void;
		variable_ptr variable;

		argument_ptr_ref( ep::variable_ptr &ptr )
			: variable( ptr )
		{ }
	};


}

EP to C++

Code:

namespace details 
	{
		template<typename..._Params>
		struct _Variable_builder;

		template<typename _Next, typename ..._Params>
		struct _Variable_builder<_Next, _Params...>
		{
			static void _Build( variable_list &list,
								method_argument_list::iterator &arg,
								_Next &&next, 
								_Params&&...rest )
			{
				list.emplace_back( new variable( (*arg)->get_name( ), std::make_shared<value_reference>( std::forward<_Next>( next ) ) ) );
				_Variable_builder<_Params...>::_Build( list, ++arg, std::forward<_Params>( rest )... );
			}
		};

		template<typename ..._Params>
		struct _Variable_builder<variable_ptr, _Params...>
		{
			static void _Build( variable_list &list,
								method_argument_list::iterator &arg,
								variable_ptr next,
								_Params&&...rest )
			{
				list.emplace_back( new variable( ( *arg )->get_name( ), next->get_ref( )->clone( ) ) );
				_Variable_builder<_Params...>::_Build( list, ++arg, std::forward<_Params>( rest )... );
			}
		};

		template<typename ..._Params>
		struct _Variable_builder<variable_ptr&, _Params...>
		{
			static void _Build( variable_list &list,
								method_argument_list::iterator &arg,
								variable_ptr &next,
								_Params&&...rest )
			{
				list.emplace_back( new variable( ( *arg )->get_name( ), next->steal( ) ) );
				_Variable_builder<_Params...>::_Build( list, ++arg, std::forward<_Params>( rest )... );
			}
		};

		template<typename _Last>
		struct _Variable_builder<_Last>
		{
			static void _Build( variable_list &list,
								method_argument_list::iterator &arg,
								_Last &&last )
			{
				list.emplace_back( new variable( ( *arg )->get_name( ), std::make_shared<value_reference>( std::forward<_Last>( last ) ) ) );
			}
		};

		template<>
		struct _Variable_builder<variable_ptr&>
		{
			static void _Build( variable_list &list,
								method_argument_list::iterator &arg,
								variable_ptr &last )
			{
				list.emplace_back( new variable( ( *arg )->get_name( ), last->get_ref( ) ) );
			}
		};

		template<>
		struct _Variable_builder<variable_ptr>
		{
			static void _Build( variable_list &list,
								method_argument_list::iterator &arg,
								variable_ptr last )
			{
				list.emplace_back( new variable( ( *arg )->get_name( ), last->get_ref( )->clone( ) ) );
			}
		};

		template<>
		struct _Variable_builder<>
		{
			static void _Build( variable_list &list,
								method_argument_list::iterator &arg )
			{ }
		};

		template<typename T>
		struct _Function_builder;

		template<typename _Ret, typename ..._Params>
		struct _Function_builder<_Ret( _Params... )>
		{
			using fn_type = std::function<_Ret( _Params... )>;
			static fn_type
				_Build( method_ptr method, dispatch_engine_ptr engine )
			{
				return [method, engine]( _Params&&...args )->_Ret
				{
					variable_list arguments;
					_Variable_builder<_Params...>::_Build( arguments,
														   method->get_arguments( ).begin( ),
														   std::forward<_Params>( args )... );
					auto result = method->get_fun( )( std::move( arguments ), engine );
					return result->assume<_Ret>( );
				};
			}
		};

		template<typename..._Params>
		struct _Function_builder<void( _Params... )>
		{
			using fn_type = std::function<void( _Params... )>;
			static fn_type
				_Build( method_ptr method, dispatch_engine_ptr engine )
			{
				return [method, engine]( _Params&&...args )->void
				{
					variable_list arguments;
					_Variable_builder<_Params...>::_Build( arguments,
														   method->get_arguments( ).begin( ),
														   std::forward<_Params>( args )... );
					method->get_fun( )( std::move( arguments ), engine );
				};
			}
		};

		template<typename..._Params>
		struct _Function_builder<value_reference( _Params... )>
		{
			using fn_type = std::function<value_reference( _Params... )>;
			static fn_type
				_Build( method_ptr method, dispatch_engine_ptr engine )
			{
				return [method, engine]( _Params&&...args )
				{
					variable_list arguments;
					_Variable_builder<_Params...>::_Build( arguments,
														   method->get_arguments( ).begin( ),
														   std::forward<_Params>( args )... );
					return method->get_fun( )( std::move( arguments ), engine );
				};
			}
		};

		template<typename _Sig>
		struct _Method_caller;

		template<typename _Ret, typename ..._Params>
		struct _Method_caller<_Ret( _Params... )>
		{
			static _Ret _Do_call( method_ptr method, dispatch_engine_ptr engine, _Params&&...params )
			{
				variable_list arguments;
				_Variable_builder<_Params...>::_Build( arguments,
													   method->get_arguments( ).begin( ),
													   std::forward<_Params>( params )... );
				auto result = method->get_fun( )( arguments, engine );
				return result->assume<_Ret>( );
			}
		};

		template<typename ..._Params>
		struct _Method_caller<void( _Params... )>
		{
			static void _Do_call( method_ptr method, dispatch_engine_ptr engine, _Params&&...params )
			{
				variable_list arguments;
				_Variable_builder<_Params...>::_Build( arguments,
													   method->get_arguments( ).begin( ),
													   std::forward<_Params>( params )... );
				method->get_fun( )( arguments, engine );
			}
		};

		template<typename ..._Params>
		struct _Method_caller<value_reference( _Params... )>
		{
			static value_reference _Do_call( method_ptr method, dispatch_engine_ptr engine, _Params&&...params )
			{
				variable_list arguments;
				_Variable_builder<_Params...>::_Build( arguments,
													   method->get_arguments( ).begin( ),
													   std::forward<_Params>( params )... );
				return method->get_fun( )( arguments, engine );
			}
		};
	}

Sorry if my explanations aren't that good, I've never been the explainy type. Feel free to add me on Discord (Yemiez#8439) for further questions!

Originally Posted by Yamiez

...

I don't see how that part can tell you it's compiled

You read a line of code and optimize it, then you reread it and interpret it: it's how classic compilers work. This is exactly the opposite which scripting languages like Python and others (probably lua too, etc) do: they immediately interpret the code you write. The method you use it's probably rare in scripting languages: it won't be so performant at all, as you take more time before interpreting it.

an interpreted language can have just the same optimizations as a compiled language

This is not true. A compiled language has the possibility to have more powerful optimizations compared to interpreted languages, and a compiler does some optimizations which are impractical from a runtime interpreter.

I haven't been very clear as to why this is, this will only happen when you use the protected ep (*.oep or obfuscated expression parser code, or short oep) format. And as of right now, the "compiler" (It never compiles into machine code!) hasn't been released yet, or atleast newer version. It creates these temporaries because the oep formats ability to store .dll's and ep script code.

Why are you calling it a compiler if it actually isn't?

It's nothing about complex, but it may be an interesting and useful (from a didactic point of view) project.

Sorry for the late reply, I've been quite busy.

Originally Posted by javalover

It isn't rare at all, my code is compiled into blocks that can be evaluated at run-time, so you can compile a single method and you never have to compile it again until you rerun the program. There only needs to be one compilation in total. (Which increases performance in comparison to other scripting languages that needs to reinterpret the code every time you want to call a function)

Originally Posted by javalover

They CAN have pretty much the same optimizations, I didn't say they were practical however.

Originally Posted by javalover

Why are you calling it a compiler if it actually isn't?

Because that's still the definition of a compiler, it just isn't a machine code compiler.
----------------------------------------------------------------------------------------------------------------------

Possible update coming soon, been very busy so haven't had time, new features:

Lambda/Anonymous methods

Example codes:

With arguments

Code:

import ( bo, ios );

method invoke( &lambda )
{
    lambda( "From invoke" );
};

invoke( method ( &from ) { cout << from << endl; } );

Without arguments

Code:

import ( bo, ios );

method invoke( &lambda )
{
    lambda( "From invoke" );
};

// either works
invoke( method { cout << "No arguments!" << endl; } );
invoke( method () { cout << "No arguments!" << endl; } );

New ways to type an identifier, any characters are allowed

Code:

import ( bo, ios );

method 'a function with spaces and stuff'( 'a lambda with spaces' )
{
	'a lambda with spaces'( );
};

'a function with spaces and stuff'( method { cout << "Hi" << endl; } );

Dynamic objects

This probably should've been implemented a while ago, lol.

Code:

import ( bo, ios );

method 'a function with spaces and stuff'( 'a lambda with spaces' )
{
	'a lambda with spaces'( );
};

dyn dynamic;
dynamic->'()' = method { cout << "operator () called!" << endl; };

// now we can call a function just passing the dynamic objects
'a function with spaces and stuff'( dynamic );

A lot more optimizations (Console window shows what the code looks like after optimization)

Oh and constant folding actually works now, it appears it has been broken for quite a while.

Originally Posted by Yamiez

...

That's not bad, but I didn't read you wrote this (if you actually did). It's however something that many interpreted languages do, not something new. For example, look at Python's peephole optimizations: https://hg.python.org/cpython/file/9...hon/peephole.c

They CAN have pretty much the same optimizations, I didn't say they were practical however.

Code:

s = ''
string.replace(s, s)

isn't able to optimize this code (don't contradict me: I'm not talking of workarounds, and if you are going to reply yes and going to show an example of applying a peephole optimizer or alike, the answer is always no - you are still doing a workaround which isn't considerable typical as it will consist of a simple searching for specific patterns of opcodes and replacing them individually with the more efficient ones). Will a traditional compiler optimize this code? Yes, it will. Just compile it and watch the native code.
While compiled languages can do a bunch of work in compile time, an interpreted one require a bunch of work at runtime. If your affermation was right, it was also right that the compiler's optimization for the performance was the same of the interpreter's one, in that case everyone would have chosen Python over C++.

Because that's still the definition of a compiler, it just isn't a machine code compiler.

For compiler we often mean a program which converts a programming language to object code, so to avoid misinterpretations there is a fantastic more-specific word you can use: interpreter.

- - - Edited - - -

https://en.wikipedia.org/wiki/Peephole_optimization

Originally Posted by javalover

I'm pretty sure I haven't written it anywhere, no. I'm pretty busy and don't have the time to make a scripting language, verbal language, school work, and the documentation. Sometimes I have to focus on other things, it's not every day I'm feeling like working on my scripting language, or any of my other projects. I do a lot of things other than programming on my freetime ^^

Originally Posted by javalover

Again, incorrect. The keyword isn't CAN, but COULD - in conditional. Can = it is always capable to do the same optimizations of a compiler; could = it might in some cases be capable to do the same optimizations of a compiler. I will do a very basic example - a traditional interpreter:

Code:
s = ''
string.replace(s, s)
isn't able to optimize this code (don't contradict me: I'm not talking of workarounds, and if you are going to reply yes and going to show an example of applying a peephole optimizer or alike, the answer is always no - you are still doing a workaround which isn't considerable typical as it will consist of a simple searching for specific patterns of opcodes and replacing them individually with the more efficient ones). Will a traditional compiler optimize this code? Yes, it will. Just compile it and watch the native code.
While compiled languages can do a bunch of work in compile time, an interpreted one require a bunch of work at runtime. If your affermation was right, it was also right that the compiler's optimization for the performance was the same of the interpreter's one, in that case everyone would have chosen Python over C++.

I suppose you're correct, excuse my ignorance! I appreciate your correction, and feedback.

Originally Posted by javalover

I understand your concern, but I do prefer the word compiler over interpreter. When I do finish (if I ever do) the language, I'll make sure to correct all of these mistakes. You have to take in mind, I'm not a professional, nor am I educated within the computer science fields. I simply enjoy it as a hobby.

Thanks!

Expression Parser Release [2x0]

Post a Reply

Similar Threads

Tags for this Thread