This is a voting example written in Viper (based on Solidity docs example). I've added it to the docs locally but would love to get some eyes on the code to make sure it's right before making a PR for the docs. @pdaian I used the same style for testing that you did, it definitely sped things up. Thanks a lot! @fubuloubu Could you take a look?

Preamble

VIP: <to be assigned>
Title: reduce the cost and difficulty of code reuse in common situations
Author: maurelian
Status: Draft
Created: 2018-06-16
Requires (*optional): <VIP number(s)>
Replaces (*optional): <VIP number(s)>

Simple Summary

1. I shouldn't need to copy and paste the same code snippets into 5 functions to valid authorization.
2. I shouldn't feel like copying and pasting code is more efficient or cheaper.

Abstract

In order to call a within the same contract function, Vyper uses the CALL opcode, which send a new message to call functions within the same contract. This has some nice safety benefits.

Unfortunately it will result in developers using anti-patterns to save on gas, or access msg.sender in a function call.

Motivation

The benefit of using CALL to access code in the same contract is to create a new execution context, with no risk of side effects from memory access. I admit that I find this to be quite an elegant use of the EVM for safety.

Problems with using CALL

Unfortunately, there are two major issues with the use of the CALL opcode for calling functions within the contract:

1. Environment values change unexpectedly

At least two important environment opcodes will return different values: CALLER (ie. msg.sender) and CALLVALUE (ie. msg.value). This complicates the use of functions for permission checking. The following is a naive vyper translation of the common "ownable" pattern for a simple storage solidity contract.

owner: address
value: public(int128)

@public
def __init__():
    self.owner = msg.sender

@private
def checkOwner():
  assert msg.sender == self.owner # this will REVERT every time

@public
def writeValue(_value: int128):
    self.checkOwner()
    self.value = _value

2. Gas costs disincentivizes code reuse

The CALL opcode costs 700 gas (+ input data costs + function dispatching again). By contrast Solidity uses JUMP (2 gas) to call a function.

This imposes a large penalty on code reuse. Regardless Vyper's design philosophy favoring safety over gas efficiency, developers will respond to this incentive by simply copying and pasting boilerplate code. This is dangerous, and difficult to audit.

Specification:

I'm sorry, I don't have one specification. Below are some options I see for mitigating these issues.

From the Motivation section above, we have 3 parameters along which to analyze approaches to calling a function in the EVM.

1. Gas cost
2. Preservation of env variables
3. Safety benefits

1. Use DELEGATECALL instead of CALL

1. Gas cost: NEUTRAL (same as using CALL)
2. Env variables: GOOD (gives us back CALLER and CALLVALUE)
3. Memory Safety: GOOD. It does indeed reset memory.

Seems like a decent solution!

~3. Memory Safety: BAD (executes the code in the same memory context, nullifying reasons for using CALL in the first place)~ ~I proposed this earlier, but I don't think this is a good solution. ~

2. Just use JUMP like Solidity

1. Gas cost: GOOD (Much lower gas cost)
2. Env variables: GOOD (Preserves CALLER and CALLVALUE)
3. Memory Safety: BAD (Greater risk of compiler bugs)

This is worth considering. See also #330.

3. Implement a safer analog to Solidity's modifiers

These could have significant restrictions on them, like no access to MSTORE or SSTORE, and not accepting arguments. I assume this would be done by with JUMP, or just inlining the same code each time it's needed.

1. Gas cost: GOOD (Lower execution cost, but might increase size of contract if inlining is used)
2. Env variables: GOOD (Preserves CALLER and CALLVALUE)
3. Memory Safety: Depends on implementation

4. Create special variables which persist across message calls

Built-in vars like caller and callvalue can be created, which are magically passed to a function in a message call.

My code above would thus be:

@private
def checkOwner():
  assert _sender == self.owner # <- replaced msg.sender with a magical global variable `sender`

@public
def writeValue(_value: int128):
    self.checkOwner)
    self.value = _value

This feels a bit funky to me. Probably would be hard to implement safely, and obscures the true functioning of the EVM.

5. Status Quo

My problem in the code sample above can be addressed by passing the values I need as arguments:

@private
def checkOwner(_sender: address):
  assert _sender == self.owner # <- replaced msg.sender with _sender

@public
def writeValue(_value: int128):
    self.checkOwner(msg.sender) # <- pass msg.sender as an argument 
    self.value = _value

Maybe that's OK? But it requires a deeper understanding of how both the EVM and Vyper work.

Backwards Compatibility

Dependent on approach.

Copyright

Copyright and related rights waived via CC0

Challenge

Who has some design chops and wants to make us a cool logo?

Solidity has a logo, but we're cooler so we should have a better one!

P.S. we :hearts: you Solidity

Rules

If you have a submission, post it to this issue as an embedded image (hosted from wherever you animals host things) and whomever has the most :+1:'s wins the bounty! Anyone can vote on submissions !

Challenge ends with our beta release (currently June 1st), where the winning submission will be added as a PR to our README/docs.

Closing Date: 1st June 2018

NOTE: Please do not submit a PR yet! Add your submission as a comment below!

NOTE: Please nothing offensive. Keep it clean, Vyperinos!

NOTE: Keep it original. As pointed out in the comments existing Vyper logos do already exist. Please refrain from making them look too similar, we don't want to get involved in any copy infrigement battles :stuck_out_tongue_closed_eyes:

NOTE: Please also see Ethereum Branding Guidelines

Sponsors

SNT bounty sponsored by Status (https://openbounty.status.im). ETH Bounty sponsored by @jacqueswww and myself Feel free to add more bounties if you desire! ETH/ERC20/ERC223 accepted

Brainstorming and Comments

Feel free to add notes with your concepts of what Vyper is and brainstorm concepts.

My design keywords (just for brainstorming):

• Snek (obvs)
• Sharp
• Secure
• Emerald
• Friendly

And some additional insperation: https://docs.google.com/document/d/1TSqVXVXVjiVxH5COXgIu90_dSJTQv3_Ka-aI6MYqeIo

NOTE: Anyone can add a comment here just chatting about the designs or whatnot. Official entries should just contain the picture (optionally some comment about it) and are voted on ONLY using :+1:

What I did

refactor type system. redo the type hierarchy - instead of distinction between "Primitives" and "Definitions", have single type system + ExprInfo

• simplify type hierarchy
• ~merge old and new type systems~ will do this in a follow up PR
• refactor builtins dir structure. merge builtin_functions/ and builtin_interfaces/ to builtins/.

How I did it

rethinking of the approach; don't need two classes for each type, factor into VyperType + ExprInfo + VarInfo. handling of types in the regular namespace is generally by dispatching into special constructor (for structs and interfaces) or attributes (for enums) mode.

How to verify it

Commit message

This commit refactors the front-end type system.

Currently, there are essentially two classes types for each vyper type.
"Primitives" represented raw types, and "Definitions" represent
the instantiation of types attached to each expression in a program.
"Definitions" additionally carry around annotation information related
to the expressions they tag, such as mutability and location info. This
system also handled dispatching into the correct routines for cases
where a raw type (e.g. "MyStruct") is used in expr land (vs an instance
of a type, e.g.  "my_struct"). This is important because certain types
are callable or attributable - specifically, constructors and enum
members (ex. `MyStruct()` and `MyEnum.FOO`).

This commit reworks the system by factoring the annotation info into
VyperType and ExprInfo. "Primitives" in the old system can be thought of
as VyperTypes, and "Definitions" in the old system are most similar to
the new ExprInfo. To handle cases where types can live in expr land,
this commit also clarifies that usage by promoting raw types into the
special, internal TYPE_T type (which can be thought of as "the type of
a type"). This commit also simplifies the inheritance structure of vyper
types.

Some other miscellaneous things in this commit:
- AST class hierarchy gets more refined - addition of ExprNode
- rename of `vyper/semantics/validation/` -> `vyper/semantics/analysis/`
- type system directory structure simplified
- merge of `builtin_functions/` and `builtin_interfaces/` to `builtins/`

Co-authored-by: z80 <[email protected]>

Description for the changelog

Cute Animal Picture

Preamble

VIP: <to be assigned>
Title: assertions that should never-ever fail
Author: Yoichi Hirai <[email protected]>
Type: Standard Track
Status: Draft
Created: 2018-03-21

Simple Summary

Add a special type of assertion that should never ever fail. When a static analyzer can fire it, the program surely has a bug.

Abstract

Static analyzers can detect bugs, but only when the desired properties are specified. The easiest way is to insert a "never-to-fail" assertions in the program.

Motivation

Mythril has a feature to detect when 0xfe is somehow reachable. KEVM can soon do that.

assert in vyper is expected to fail sometimes, and it is compiled to REVERT. REVERT might indicate a mistake of the caller, a mistake of the programmer, or a mistake of the compiler. So, static analyzers cannot shout "this is a bug".

Instead, if Vyper has a never-to-fail assertion that translates to INVALID (0xfe), static analyzers can confidently shout "this is a bug. See, this execution reaches INVALID".

Alternatives are specially formatted comments like ACSL or JML, but they are hard to learn.

Specification

Add a additional custom constant to the assert statement:

    assert amount != 0, UNREACHABLE

as well as

    raise UNREACHABLE

Backwards Compatibility

A program containing a name assure will cause compillation errors.

Copyright

Copyright and related rights waived via CC0

What's your issue about?

We don't have sqrt function, this is useful to have for bonding curves.

How can it be fixed?

Embed this code:

def test_sqrt(get_contract):

    code = """
@public
def sqrt256(x: uint256) -> uint256:
    z: uint256 = (x + 1) / 2
    y: uint256 = x

    for i in range(256):
        if (z > y):
            break
        y = z
        z = (x / z + z) / 2
    return y


@public
def sqrt_decimal(x: decimal) -> decimal:
    assert x >= 0.0
    if x == 0.0:
        return 0.0
    z: decimal = (x + 1.0) / 2.0
    y: decimal = x

    for i in range(256):
        if (z == y):
            break
        y = z
        z = (x / z + z) / 2.0
    return y

    """

    c = get_contract(code)

    assert c.sqrt256(2) == 1
    # math.sqrt(2000) == 44.721359549995796
    assert c.sqrt_decimal(2000) == Decimal('44.7213595499')

When working with the Eth2 deposit contract I noticed a strange behaviour which I presume is a compiler bug.

Notice that the tests pass in https://github.com/ethereum/eth2.0-specs/pull/1152/commits/abe48cc98857c3350d45656d3d4a6f2e6af2784d (the command to run the tests is make compile_deposit_contract; make test_deposit_contract) but fail if you move line 92 (the Deposit log) to line 80 the tests fail when nothing has really changed. I haven't had time to debug it but my guess would be that that it has something to do with zero_bytes32: bytes32, possibly that junk is being written to it.

For ABI-formatted logs, we'd need to have syntax for both event type declaration and making the actual logs. For event type declaration, one option is for the parser to expect a list of declarations between storage variable declarations and function declarations that look like this:

MyLog: __log__(arg_1: num, arg2: indexed(bytes32), arg3: timestamp, arg4: bytes <= 100)

For the actual logging, I personally highly disagree with Solidity's syntax of making event logs and function calls syntactically the same; it goes against Viper's goals of making it hard to write misleading code.

One option is to copy Serpent syntax:

log(type=MyLog, 5, sha3("cow"), block.timestamp, "moose")

Another is:

log.MyLog(5, sha3("cow"), block.timestamp, "moose")

For external contract calls, we already have raw_call, but it would be good to have something ABI-formatted. We have the ABI formatting call in place already for self-calls, but to extend that to external calls we need a syntax for declaring external contract types.

One option is for the parser to expect another list of declarations, like:

class Foo():
    def bar(arg1: num, arg2: bytes32): pass
    @constant
    def baz(arg1: bytes <= 100, arg2: num256): pass

Another is to just directly pass in JSON ABI:

Foo = extern([{"constant": false, "type": "function", "name": "bar(int128,bytes32)", "outputs": [], "inputs": [{"type": "int128", "name": "arg1"}, {"type": "bytes32", "name": "arg2"}]}, {"constant": true, "type": "function", "name": "baz(bytes,uint256)", "outputs": [], "inputs": [{"type": "bytes", "name": "arg1"}, {"type": "uint256", "name": "arg2"}]}])

Then, one could declare anything from then on as having type Foo (we'd require contract types to start with a capitalized letter).

Any other ideas?

This is a tracking issue that I'm using to dump my thoughts on things that can be cleaned up in this codebase.

General

• [x] convert to circle-ci
• [x] make codebase pep8 compliant
• [x] add mypy
• [x] use tox
• [ ] use enum instead of strings for typ values.
• [ ] use docstrings instead of comments for class definitions in vyper.types.types
• [x] Use functools.wraps for decorators
• [ ] Use proper semantically meaningful exception classes instead of the base Exception
• [ ] Create Opcodes Enum for easier referencing opcodes.
• [ ] Use of floating point values seems like a foot gun. Maybe use decimal.Decimal?
• [ ] Use f-strings
• [ ] Use of magic string in location == "memory" should be an Enum.
• [ ] Rename sha3 to keccak across codebase.

Specifics

• [ ] vyper.type.types.NodeType.__eq__

change NodeType.__eq__ to require subclasses to implement and remove eq hook.

• [ ] vyper.types.types.parse_type

Takes multiple arguments like sigs, custom_units, custom_structs and constants. Current read of the code suggests there are data structures which have been defined in the code and thus, this gives the parser the ability to properly parse the AST into these types. If so, I believe this should be formalized into something akin to the WASM Store to wrap these data structures up into a single wrapper for easier and less error prone portability.

• [ ] vyper.types.types.StructType

Takes a mapping of field_name -> field_type for members argument. This relies on dictionary ordering (which is admittedly now reliable) but also is a more difficuult data structure to reliably type hint. Consider using an iterable of 2-tuples: (('field_a', TypeA), ('field_b', TypeB)). Benefits are easier type safety through type hinting. Cons are additional validation step to ensure field uniqueness.

• [ ] vyper.types.types.get_size_of_type

Relies on prior knowledge about types. This should probably be an API of the type classes.

• [ ] vyper.types.types.has_dynamic_data

Relies on prior knowledge about types to determine. This should probably just be an API of the type classes.

• [ ] vyper.functions.signature.signature

Uses non-canonical argz and kwargz with z suffixes. Convert to normal *args, **kwargs

Inner decorator function isn't using functools.wraps.

More usage of magic strings where an Enum is probably better.

• [ ] vyper.opcodes

Replace use of list with NamedTuple.

• [ ] vyper.compiler.gas_estimate

Uses runtime assert statement which is easily disabled. Change to explicit exception raising.

• [x] vyper.compiler.mk_full_signature

Loop uses enumerate to generate indices idx used to modify abi at that indices. Instead, use zip to pair functions with their abi definitions.

• [ ] vyper.compiler.get_source_map

Builds dict using mutation. Vendor a version of to_dict decorator to remove mutation pattern.

• [x] vyper.compiler.compile_codes

Uses mutable data structure as default for argument

Use of magic strings for output_type should be constrained to enums.

• [x] vyper.compiler.compile_code

Uses mutable data structure as default for argument

• [ ] vyper.compiler.get_asm

constant string mutation (which actual ends up as full object copies) adds overhead. Converting to a wrapped generator pattern that joins all of the produced strings at the end should be more performant and remove mutation.

• [ ] vyper.compile_lll

Use of next_symbol global counter appears to be used to create a unique marker in the code such as _sym_0, _sym_1. Replace with itertools.count(). Replace with uuid.uuid4(). Replace with generator function that wraps an itertools.count() style counter. Embed within an object with a defined lifecycle that gets passed around during compilation.

• [ ] vyper.compile_lll.is_symbol

Replace use of magic string with something more concrete like a custom class. Potentially the custom class could handle the mechanism for generating unique symbols.

• [ ] vyper.compile_lll.instruction

instruction is a class and thus should be named using common convention Instruction.

• [x] vyper.compile_lll.apply_line_numbers

Needs to use functools.wraps for inner function.

• [ ] vyper.compile_lll.compile_to_assembly

Convert runtime assert statements to raise proper exceptions.

• [ ] vyper.compile_lll.compile_to_assembly

Detection of whether something is or is not an opcode is done via isinstance(value, str) and value in opcodes. If we had an Opcodes Enum this could be as simple as checking if Opcodes(value) raised an exception or not.

• [ ] vyper.compile_lll.compile_to_assembly

Use of inlined opcode string values like "PUSH" would be better as Enum or constant values.

• [ ] vyper.compile_lll.compile_to_assembly

Individual blocks in the if/elif/.../else block could be partitioned off into smaller standalone functions.

• [ ] vyper.compile_lll.compile_to_assembly

The section that handles clamping values has an if/elif which is missing a final else clause allowing for undefined behavior if none of the conditions evaluates truthy

• [ ] vyper.compile_lll.assembly_to_evm

Use formal data structure for line_number_map instead of dictionary.

• [ ] vyper.compile_lll.assembly_to_evm

Inside of the if isinstance(item, list), the local value line_number_map is overwritten which appears to be a potentially destructive mistake as it will destroy any state that was previously built up locally.

• [ ] vyper.optimizer.get_int_at

Uses inline logic for converting a value to a signed integer. Should be extracted into utility that is tested.

• [ ] vyper.util.is_instances

Confusingly named. Very visually similar to isinstance. Maybe rename to a all_is_instances

• [ ] vyper.parser.constants.Constants.unroll_constant

Use of fail flag is antipattern. Restructure to allow inlined exception raising.

• [x] vyper.parser.context.Context.start_blockscope and others

The various function pairs like start_blockscope/end_blockscope, set_in_assertion, set_in_for_loop/remove_in_for_loop should be converted to use context manager pattern so that exiting the scope cannot be accidentally forgotten.

• [ ] vyper.parser.global_ctx.GlobalContext.get_item_name_and_attributes

Returns None for item name in final block which suggests there is either a case where something does not have a name or that there is return value checking occuring somewhere.

• [ ] vyper.parser.stmt.Statements._check_valid_assign

Does not have a final else block allowing function to exit if none of the conditions evaluate truthy.

• [ ] vyper.parser.stmt.Statements.parse_for

Uses implicit else for final clause type. Should be converted to explicitely check that the statement is formed as expected and add a final else block that raises an invariant exception.

Preamble

VIP: 563 Title: Custom Parser Author: @fubuloubu @DavidKnott @jacqueswww Type: Standard Status: Draft Created: 2017-12-08

Simple Summary

Implement a custom parser for Viper that doesn't directly tie us to Python-only syntax, enabling a more focused grammer for our langauge

Abstract

We've been discussing this for a while. A custom parser would allow us to define our syntax more precisely rather than leveraging Python syntax and being tied to only what Python's syntax can provide. We will continue to use Python as a template due to it's clarity and ease of reading, but we need to make decisions that diverge from Python and a custom Parse will enable that.

Motivation

There are specific things that have been discussed where this is necessary:

• Clarifying the external contract type
• Changing the mapping type for greater clarity
• Custom Types
• etc...

Specification

We may be able to leverage a Python-compatible lex/yacc library like ply. We should also leverage some of the work the k-framework guys are doing in order to infer a grammar that is consistent and free of formalized conflicts

Backwards Compatibility

Try to maintain backwards compatibility initially, however some of the VIPs this one will enable will be breaking changes in the syntax.

Copyright

Copyright and related rights waived via CC0

What I did

This commit implements Solidity-ABI-compatible dynamic arrays in vyper.

Dynamic arrays have the type DynArray[<type>, <maxlen>]. They are implemented similarly to bytestrings, with a single length word followed by the data, <len> <data...>.

How I did it

Threaded through typechecker, added abi encoder/decoder and added the relevant functionality in codegen

How to verify it

See tests

Description for the changelog

add dynamic arrays to vyper

Cute Animal Picture

What I did

As per #3214, trying to reference a yet undeclared constant variable in the type annotation of another variable's declaration results in a fairly unhelpful error message: vyper.exceptions.UndeclaredDefinition: 'self' has not been declared. This fix ensures that a more detailed exception is raised.

How I did it

The exception is raised when trying to check whether the variable is a storage variable by checking if its name is in self.namespace["self"].typ.members. However this check can happen before self is added to the namespace. When the constant variable is referenced in a type annotation, the check will happen during folding and before validation, so there will be no self key in namespace, leading to the previous exception being raised. A simple check on whether the key is present suffices to prevent this exception from being raised and to instead raise another detailed UndeclaredDefinition exception containing the undeclared variable's name and line number of the error.

How to verify it

Compiling the following contract:

PRECISIONS: constant(uint256[N_COINS]) = [
    1000000000000,
    10000000000,
    1,
]
N_COINS: constant(int128) = 3

Now raises the following exception:

vyper.exceptions.UndeclaredDefinition: 'N_COINS' has not been declared. 
  contract "***.vy:1", line 1:29 
  ---> 1 PRECISIONS: constant(uint256[N_COINS]) = [
  ------------------------------------^
       2     1000000000000,

Commit message

Raise clearer exception when using a yet undeclared variable in a type annotation

Description for the changelog

Cute Animal Picture

Version Information

• vyper Version (output of vyper --version): 0.3.7
• OS: linux
• Python Version (output of python --version): 3.10

What's your issue about?

Referencing a constant variable in a subscript in a type annotation before it is declared when declaring another state variable gives the following unclear error message: vyper.exceptions.UndeclaredDefinition: 'self' has not been declared.

Can be reproduced with the following example:

PRECISIONS: constant(uint256[N_COINS]) = [
    1000000000000,
    10000000000,
    1,
]

N_COINS: constant(int128) = 3

normally, type checking is idempotent, but in the case of for loop type checking, correct typechecking depends on the "type" field being fresh. this moves the type annotation to a different field which does not shadow "type".

fixes #3212

What I did

How I did it

How to verify it

Commit message

Commit message for the final, squashed PR. (Optional, but reviewers will appreciate it! Please see our commit message style guide for what we would ideally like to see in a commit message.)

Description for the changelog

Cute Animal Picture

Version Information

• vyper Version (output of vyper --version): 0.3.7

the type of a loop variable can take multiple values in a loop. minimal repro:

@external
def main():
    for j in range(3):
        x: uint256 = j
        y: uint16 = j

thanks to @trocher for reporting

What I did

fix #3093. Restricted mutability of raw_call to @view when STATICCALL is used

How I did it

How to verify it

Commit message

restricted mutability of raw_call to @view when STATICCALL is used

Functions that make use of the STATICCALL opcode must be declared as view.

Description for the changelog

Cute Animal Picture

What I did

Added an optional argument to raw_call to allow for a safety check on the size of the return data. When revert_on_excess is set to True, the call will revert if the receiving variable is smaller than the returned data. (This is the default behavior already on external calls, but I set the default value to False here).

How I did it

Added an assert comparing returndatasize and max_outsize when revert_on_excess is set to True.

How to verify it

Run the updated tests in test_raw_call.py

Commit message

Option to add an upper bound check on returndatasize with raw_call

Description for the changelog

Cute Animal Picture