elf
/
polyloader
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Sorta an initial check-in. 

This is broken out from a week-long development cycle for the
[Hy](http://docs.hylang.org/en/latest/) programming language,
which I can't recommend enough.
This commit is contained in:
Elf M. Sternberg 2016-05-29 09:42:34 -07:00
commit c508173e7d
9 changed files with 354 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
.gitignore vendored Normal file
View File

@ -0,0 +1,8 @@
*~
*.swp
db.sqlite3
bower_components
*.pyc
\#*#
.#*
catalogia/settings.hy

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2004-2016 Elf M. Sternberg <elf.sternberg@gmail.com>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

7
MANIFEST.in Normal file
View File

@ -0,0 +1,7 @@
include *.py
include LICENSE
exclude run_tests.py
exclude requirements.txt
exclude README.md
recursive-include *.py
prune tests

29
README.rst Normal file
View File

@ -0,0 +1,29 @@
_polyloader_ is a python module to hook into Python's import machinery
and insert your own syntax parser/recognizer. Importlib uses filename
suffixes to recognize which compiler to use, but is internally
hard-coded to only recognize ".py" as a valid suffix.
## To use:
Import polyloader in your python script's launcher or library, as well
as the syntax compiler(s) you plan to use. For example, if you have
[Mochi](https://github.com/i2y/mochi)) and
[Hy](http://docs.hylang.org/en/latest/) installed, and you wanted to
write a Django app, edit manage.py and add the following lines at the
top:
.. code:: python
from mochi.main import compile_file as mochi_compile
from hy.importer import ast_compile as hy_compile
from polyloader import polyimport
polyimport(mochi_compile, ['.mochi'])
polyimport(hy_compile, ['.hy'])
Now your views can be written in Hy and your models in Mochi, and
everything will just work.
## Dependencies
polymorph is self-contained. It has no dependencies other than Python
itself and your choice of language.

178
polyloader.py Normal file
View File

@ -0,0 +1,178 @@
# polyloader.py
#
# Copyright (c) 2016 Elf M. Sternberg <elf.sternberg@gmail.com>
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
#
""" Utilities for initializing extended path-hooks into the Python runtime """
__all__ = [] # Exports nothing; this module is called for its side-effects
import os
import sys
from importlib import machinery
from importlib.machinery import SOURCE_SUFFIXES as PY_SOURCE_SUFFIXES
from pkgutil import iter_importer_modules
try:
from importlib._bootstrap import _get_supported_file_loaders
except:
from importlib._bootstrap_external import _get_supported_file_loaders
__author__ = 'Elf M. Sternberg'
__version__ = '2016.05.29'
__contact__ = 'elf.sternberg@gmail.com'
def _call_with_frames_removed(f, *args, **kwds):
# Hack. This function name and signature is hard-coded into
# Python's import.c. The name and signature trigger importlib to
# remove itself from any stacktraces. See import.c for details.
return f(*args, **kwds)
class ExtendedSourceFileLoader(machinery.SourceFileLoader):
"""Override the get_code method. Falls back on the SourceFileLoader
if it's a Python file, which will generate pyc files as needed,
or works its way into the Extended version. This method does
not yet address the generation of .pyc/.pyo files from source
files.
"""
_source_handlers = []
@classmethod
def get_extended_suffixes(cls):
suffixes = []
for compiler, csuffx in cls._source_handlers:
suffixes = suffixes + list(csuffx)
return suffixes
@classmethod
def get_extended_suffixes_inclusive(cls):
return PY_SOURCE_SUFFIXES + cls.get_extended_suffixes()
# TODO: Address the generation of .pyc/.pyo files from source files.
# See importlib/_bootstrap.py for details is SourceFileLoader of
# how that's done.
def get_code(self, fullname):
source_path = self.get_filename(fullname)
if source_path.endswith(tuple(PY_SOURCE_SUFFIXES)):
return super(ExtendedSourceFileLoader, self).get_code(fullname)
for compiler, suffixes in self._source_handlers:
if source_path.endswith(suffixes):
return compiler(source_path, fullname)
else:
raise ImportError("Could not find compiler for %s (%s)" % (fullname, source_path))
# Provide a working namespace for our new FileFinder.
class ExtendedFileFinder(machinery.FileFinder):
# Taken from inspect.py and modified to support alternate suffixes.
@staticmethod
def getmodulename(path):
fname = os.path.basename(path)
suffixes = [(-len(suffix), suffix)
for suffix in (machinery.all_suffixes() +
ExtendedSourceFileLoader.get_extended_suffixes())]
suffixes.sort() # try longest suffixes first, in case they overlap
for neglen, suffix in suffixes:
if fname.endswith(suffix):
return fname[:neglen]
return None
# Taken from pkgutil.py and modified to support alternate suffixes.
@staticmethod
def iter_modules(importer, prefix=''):
if importer.path is None or not os.path.isdir(importer.path):
return
yielded = {}
try:
filenames = os.listdir(importer.path)
except OSError:
# ignore unreadable directories like import does
filenames = []
filenames.sort() # handle packages before same-named modules
for fn in filenames:
modname = ExtendedFileFinder.getmodulename(fn)
if modname == '__init__' or modname in yielded:
continue
path = os.path.join(importer.path, fn)
ispkg = False
if not modname and os.path.isdir(path) and '.' not in fn:
modname = fn
try:
dircontents = os.listdir(path)
except OSError:
# ignore unreadable directories like import does
dircontents = []
for fn in dircontents:
subname = ExtendedFileFinder.getmodulename(fn)
if subname == '__init__':
ispkg = True
break
else:
continue # not a package
if modname and '.' not in modname:
yielded[modname] = 1
yield prefix + modname, ispkg
pass
# Monkeypatch both path_hooks and iter_importer_modules to make our
# modules recognizable to the module iterator functions. This is
# probably horribly fragile, but there doesn't seem to be a more
# robust way of doing it at the moment, and these names are stable
# from python 2.7 up.
def install(compiler, suffixes):
""" Install a compiler and suffix(es) into Python's sys.path_hooks, so
that modules ending with thoses suffixes will be parsed into
python executable modules automatically.
"""
filefinder = [(f, i) for i, f in enumerate(sys.path_hooks)
if repr(f).find('path_hook_for_FileFinder') != -1]
if not filefinder:
return
filefinder, fpos = filefinder[0]
ExtendedSourceFileLoader._source_handlers = (ExtendedSourceFileLoader._source_handlers +
[(compiler, tuple(suffixes))])
supported_loaders = _get_supported_file_loaders()
sourceloader = [(l, i) for i, l in enumerate(supported_loaders)
if repr(l[0]).find('importlib.SourceFileLoader') != -1]
if not sourceloader:
return
sourceloader, spos = sourceloader[0]
supported_loaders[spos] = (ExtendedSourceFileLoader,
ExtendedSourceFileLoader.get_extended_suffixes_inclusive())
sys.path_hooks[fpos] = ExtendedFileFinder.path_hook(*supported_loaders)
iter_importer_modules.register(ExtendedFileFinder, ExtendedFileFinder.iter_modules)
if sys.path[0] !== "":
sys.path.insert(0, "")

1
requirements.txt Normal file
View File

@ -0,0 +1 @@
future

15
run_tests.py Normal file
View File

@ -0,0 +1,15 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
"""Script to run the tests."""
import sys
from unittest import TestLoader, TextTestRunner
# Change PYTHONPATH to include pefile.
sys.path.insert(0, u'.')
if __name__ == '__main__':
test_suite = TestLoader().discover('./tests', pattern='*_test.py')
test_results = TextTestRunner(verbosity=2).run(test_suite)
if not test_results.wasSuccessful():
sys.exit(1)

95
setup.py Executable file
View File

@ -0,0 +1,95 @@
#!/usr/bin/python
import ast
import os
import re
import sys
try:
from setuptools import setup, Command
except ImportError as excp:
from distutils.core import setup, Command
from unittest import TestLoader, TextTestRunner
os.environ['COPY_EXTENDED_ATTRIBUTES_DISABLE'] = 'true'
os.environ['COPYFILE_DISABLE'] = 'true'
def _read_doc():
"""
Parse docstring from file 'polyloader.py' and avoid importing
this module directly.
"""
with open('polyloader.py', 'r') as f:
tree = ast.parse(f.read())
return ast.get_docstring(tree)
def _read_attr(attr_name):
"""
Parse attribute from file 'polyloader.py' and avoid importing
this module directly.
__version__, __author__, __contact__,
"""
regex = attr_name + r"\s+=\s+'(.+)'"
with open('polyloader.py', 'r') as f:
match = re.search(regex, f.read())
# Second item in the group is the value of attribute.
return match.group(1)
class TestCommand(Command):
"""Run tests."""
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
polyloader_version = _read_attr('__version__')
if 'bdist_msi' in sys.argv:
polyloader_version, _, _ = polyloader_version.partition('-')
class TestCommand(Command):
"""Run tests."""
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
test_suite = TestLoader().discover('./tests', pattern='*_test.py')
test_results = TextTestRunner(verbosity=2).run(test_suite)
setup(name = 'polyloader',
version = polyloader_version,
description = 'Python artbitrary syntax import hooks',
author = _read_attr('__author__'),
author_email = _read_attr('__contact__'),
url = 'https://github.com/elfsternberg/py-polymorphic-loader',
keywords = ['python', 'import', 'language', 'hy', 'mochi'],
classifiers = [
'Development Status :: 3 - Alpha',
'Intended Audience :: Developers',
'Natural Language :: English',
'Operating System :: OS Independent',
'Programming Language :: Python',
'Topic :: Software Development :: Libraries :: Python Modules'],
long_description = "\n".join(_read_doc().split('\n')),
cmdclass={"test": TestCommand},
py_modules = ['polyloader'],
install_requires=[
'future',
],
)

0
tests/.keep Normal file
View File