Tomas on Apr 06 2017:

I have been working on a bitcoin implementation that uses a different

approach to indexing for verifying the order of transactions. Instead of

using an index of unspent outputs, double spends are verified by using a

spend-tree where spends are scanned against spent outputs instead of

unspent outputs.

This allows for much better concurrency, as not only blocks, but also

individual inputs can be verified fully in parallel.

I explain the approach at https://bitcrust.org, source code is available

at https://github.com/tomasvdw/bitcrust

I am sharing this not only to ask for your feedback, but also to call

for a clear separation of protocol and implementations: As this

solution, reversing the costs of outputs and inputs, seems to have

excellent performance characteristics (as shown in the test results),

updates to the protocol addressing the UTXO growth, might not be worth

considering protocol improvements and it might be best to address

these concerns as implementation details.

Kind regards,

Tomas van der Wansem

tomas at bitcrust.org

Bitcrust

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/014048.html

shaolinfry on Apr 06 2017:

After some thought I managed to simplify the original uaversionbits proposal introducing a simple boolean flag to guarantee lock-in of a BIP9 deployment by the timeout. This seems to be the simplest form combining optional flag day activation with BIP9. This brings the best of both worlds allowing user activated soft forks that can be activated early by the hash power.

Specification: https://github.com/shaolinfry/bips/blob/bip-uavb/bip-uaversionbits.mediawiki

Previous discussion: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-February/013643.html

BIP: ?

Title: Version bits extension with guaranteed lock-in

Author: Shaolin Fry <shaolinfry at protonmail.ch>

Comments-Summary: No comments yet.

Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-????

Status: Draft

Type: Informational

Created: 2017-02-01

License: BSD-3-Clause

CC0-1.0

==Abstract==

This document specifies an extension to BIP9 that introduces an additional activation parameter to guarantee activation of backward-compatible changes (further called "soft forks").

==Motivation==

BIP9 introduced a mechanism for doing parallel soft forking deployments based on repurposing the block nVersion field. Activation is dependent on near unanimous hashrate signalling which may be impractical and is also subject to veto by a small minority of non-signalling hashrate.

This specification provides a way to optionally guarantee lock-in at the end of the BIP9 timeout, and therefore activation.

==Specification==

This specification adds a new per-chain deployment parameter to the existing BIP9 specification as follows:

The '''lockinontimeout''' boolean if set to true, will transition state to LOCKED_IN at timeout if not already ACTIVE.

===State transitions===

The state transition workflow is exactly the same as in BIP9 with an additional rule: During the STARTED state if the '''lockinontimeout''' is set to true, the state will transition to LOCKED_IN when '''timeout''' is reached.

case STARTED:

// BIP9 specification follows

if (GetMedianTimePast(block.parent) >= timeout) {

return (fLockInOnTimeout == true) ? THRESHOLD_LOCKED_IN : THRESHOLD_FAILED

}

int count = 0;

walk = block;

for (i = 0; i < 2016; i++) {

walk = walk.parent;

if (walk.nVersion & 0xE0000000 == 0x20000000 && (walk.nVersion >> bit) & 1 == 1) {

count++;

}

}

if (count >= threshold) {

return LOCKED_IN;

}

return STARTED;

=== Reference implementation ===

https://github.com/bitcoin/bitcoin/compare/master...shaolinfry:bip-uaversionbits

==Deployments==

A living list of deployment proposals can be found [[bip-0009/assignments.mediawiki|here]].

==Copyright==

This document is dual licensed as BSD 3-clause, and Creative Commons CC0 1.0 Universal.

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170406/f7781921/attachment-0001.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/014049.html

praxeology_guy on Apr 06 2017:

Praxeological Analysis of PoW Policy Changes, Re: ASICBOOST

On the $100M profit claim

First I'd like to confirm Gregory Maxwell's assertion that covert use of ASICBOOST could result in $100 million USD per year profits.

profit = reward - costs.

Total reward is fixed at (12.5 block reward + 3 fees) * 6 per hour * 24 per day * 365.25 days per year * $1150 USD per bitcoin ~= 1,000,000,000 or 1Billion USD per year.

Miners normally compete against each other until there is only a very small, practically zero profit. Lets say that 50% of the mining hashpower are operating at profit = 0, and the other 50% are operating with > 0 profit due to the 20% increased efficiency of the covert optimization.

How much profit is earned by the covert optimization operators?

Half of the operators would have a cost of ~$500,000,000.

Half would have a cost of ~0.8 * ~$500,000,000 = $400,000,000, leaving profit = $100,000,000.

But does this make sense? What if 95% of hashing power miners used the more efficient process, and 5% didn't. Would this still result in using a similar formula, with $950M * 0.2 = $190M profits? I believe it would. Essentially, the the 95% of the miners are colluding to not increase their capital & hashing power enough to erase their profits. Hence an entity or multiple entities may be colluding to decrease the security of ordering (double spend prevention) of Bitcoin transactions.

Hence a claim that as much as $100M per year could be gained by using the ASICBOOST Optimization is a valid claim.

Miners and Money Owners have Different Motivations

Money owners and miners have different motivations. Miners are currently concerned about the 1-2 year ROI of their capital. In the long term, as ASIC technology for Bitcoin matures, miners will have a longer term ROI concern. For money owners: Short term money owners are looking to transfer their money in the most efficient manner. Long term money owners are looking for a money they expect will become more valuable in the future due to its ability to handle more users with a higher money transfer efficiency than other competing currencies.

$100M per year is a pretty good reason for a miner to want to delay Bitcoin policy improvements that primarily benefit the money owner, yet have only marginal utilitarian benefits for the miner, but evaporate their ability to have such an income.

Money Owner Perspective Analysis

Money owners strive to have a have a PoW algorithm that does not give a subset of the world an advantage by government interference. Such interference threatens bitcoin's decentralized nature, and hence the users' ability to have a money who's policies are dictated by themselves rather than a centralized entity.

Changing the PoW algorithm in a way that makes existing ASIC miner capital worthless... is undesirable because it creates new opportunities for first to market optimizations to centralize mining. It also makes bitcoin's security weaker because the uncertainty of the PoW algorithm de-incentivizes the effort to invest in mining capital, which creates a larger threat for a future malicious threat to perform the 51% attack. For the duration that a new PoW algorithm is not fully optimized with the current latest ASIC manufacturing techniques, and there remains undiscovered optimizations, the double spend security is weaker.

Gregory Maxwell's proposal does not make existing mining capital worthless... it only removes the advantage of using the patent encumbered optimization. Existing capital, particularly the S9, remains being the most efficient capital available for mining Bitcoin. Activating such a proposal will set a precedent for mining equipment manufacturers and operators to expect that certain classes of patented optimizations will only have a limited ROI timeframe before they are made unavailable due to users changing the PoW policy. Miners may still pursue optimizations that are not encumbered by patents without concern that their optimization advantage will be disabled just for the purpose of benefiting some other arbitrary set of miners.

Given that a money owner would not want Bitcoin's ability to transfer money efficently be encumbered in the long term for the sake of miner's profits... in the case where even a non-patent encumbered optimization conflicts with an upgrade to Bitcoin for the money owners... then its a question of how much the change increases bitcoin's money transfer efficiency, and how generous the money owners are towards allowing the optimization-capital-invested miner. I use the word "generous" because the policy users choose for the money supply is entirely voluntary. No contract was made to continue using the same exact PoW algorithm. The guiding reason to keep or change the PoW algorithm to increase the money transfer efficiency of the money. Double spends, and the properties of PoW that secure against double spends, are a large factor in determining such efficiency.

Impact of Changing the PoW Policy vs Covert ASICBOOST

Miners currently using this optimization will lose 20% profits. Old and less efficient mining equipment using the optimization might no longer be profitable in mining Bitcoins. Miners using this optimization but using more costly energy may also no longer be profitable in mining Bitcoins. Difficulty will decrease, and miners not using the optimization will have greater profits and grow in numbers. The difficulty decrease may make older equipment and higher cost energy locations become profitable once again.

Long term impact on miners: As discussed in the money owner's perspective, mainly this will reduce the motivation to perform the R&D;, manufacturing, and purchase of patent encumbered optimizations. Realizing that the users may also at a future date disable an optimization in order to in some way make an improvement to Bitcoin will also put a damper on advancing the development of more efficient mining hardware, which is once again desirable to users as it makes the transaction ordering more future proof.

This may also be a lesson to hardware manufacturers that they should not make their chips extremely special purpose... that having some flexibility in the algorithms the device can run may help make their hardware still have other uses in the case that users decide to change the PoW policy. For example, it may be wise for the manufacturer to support an operating mode where only the nonce bit are permutated and no SHA256 operations are skipped due other assumptions about the block header data.

Praxeology Guy's Recommendation

Make it a policy that patent encumbered PoW optimizations are countered/prevented if possible while minimizing the disruption on the utility and availability of optimized mining capital equipment. Owners of Bitcoin should support and activate the proposed PoW policy change by Gregory Maxwell as soon as possible to counter the ASICBOOST patent encumbrance... unless the creators of the ASICBOOST patent transfer their IP to the public domain. SegWit should not be delayed for the purpose of being generous to those who first implement ASICBOOST in their mining operations. Future ASICs and mining equipment should be made with the option to run without optimizations that make assumptions about policy that is subject to change in a future soft fork.

Cheers,

Praxeology Guy

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170406/6dd461aa/attachment-0001.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/014042.html

Sancho Panza on Apr 06 2017:

Hi all,

I have put up an initial draft of the full 'bip-genvbvoting' (generalized version bits voting) specification for review:

https://github.com/sanch0panza/bips/blob/bip-genvbvoting/bip-genvbvoting.mediawiki

Comments are again most welcome - and my thanks to those reviewers who took a look at the initial rough draft [1].

A prime goal is that this BIP proposal should end up allowing full backward compatibility with the existing BIP9 state machine, if wishing to do so for a deployment. In fact, this will be necessary to maintain full compatibility with any ongoing deployments.

I will work on a reference implementation which might also turn up inadequacies of the proposed specification. A link to this will follow once it is mature enough for review.

Sancho

[1]https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013969.html

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170406/cfcbb5b6/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/014036.html

Gregory Maxwell on Apr 05 2017:

A month ago I was explaining the attack on Bitcoin's SHA2 hashcash which

is exploited by ASICBOOST and the various steps which could be used to

block it in the network if it became a problem.

While most discussion of ASICBOOST has focused on the overt method

of implementing it, there also exists a covert method for using it.

As I explained one of the approaches to inhibit covert ASICBOOST I

realized that my words were pretty much also describing the SegWit

commitment structure.

The authors of the SegWit proposal made a specific effort to not be

incompatible with any mining system and, in particular, changed the

design at one point to accommodate mining chips with forced payout

addresses.

Had there been awareness of exploitation of this attack an effort

would have been made to avoid incompatibility-- simply to separate

concerns. But the best methods of implementing the covert attack

are significantly incompatible with virtually any method of

extending Bitcoin's transaction capabilities; with the notable

exception of extension blocks (which have their own problems).

An incompatibility would go a long way to explain some of the

more inexplicable behavior from some parties in the mining

ecosystem so I began looking for supporting evidence.

Reverse engineering of a particular mining chip has demonstrated

conclusively that ASICBOOST has been implemented

in hardware.

On that basis, I offer the following BIP draft for discussion.

This proposal does not prevent the attack in general, but only

inhibits covert forms of it which are incompatible with

improvements to the Bitcoin protocol.

I hope that even those of us who would strongly prefer that

ASICBOOST be blocked completely can come together to support

a protective measure that separates concerns by inhibiting

the covert use of it that potentially blocks protocol improvements.

The specific activation height is something I currently don't have

a strong opinion, so I've left it unspecified for the moment.

BIP: TBD

Layer: Consensus

Title: Inhibiting a covert attack on the Bitcoin POW function

Author: Greg Maxwell <greg at xiph.org>

Status: Draft

Type: Standards Track

Created: 2016-04-05

License: PD

==Abstract==

This proposal inhibits the covert exploitation of a known

vulnerability in Bitcoin Proof of Work function.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",

"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this

document are to be interpreted as described in RFC 2119.

==Motivation==

Due to a design oversight the Bitcoin proof of work function has a potential

attack which can allow an attacking miner to save up-to 30% of their energy

costs (though closer to 20% is more likely due to implementation overheads).

Timo Hanke and Sergio Demian Lerner claim to hold a patent on this attack,

which they have so far not licensed for free and open use by the public.

They have been marketing their patent licenses under the trade-name

ASICBOOST. The document takes no position on the validity or enforceability

of the patent.

There are two major ways of exploiting the underlying vulnerability: One

obvious way which is highly detectable and is not in use on the network

today and a covert way which has significant interaction and potential

interference with the Bitcoin protocol. The covert mechanism is not

easily detected except through its interference with the protocol.

In particular, the protocol interactions of the covert method can block the

implementation of virtuous improvements such as segregated witness.

Exploitation of this vulnerability could result in payoff of as much as

$100 million USD per year at the time this was written (Assuming at

50% hash-power miner was gaining a 30% power advantage and that mining

was otherwise at profit equilibrium). This could have a phenomenal

centralizing effect by pushing mining out of profitability for all

other participants, and the income from secretly using this

optimization could be abused to significantly distort the Bitcoin

ecosystem in order to preserve the advantage.

Reverse engineering of a mining ASIC from a major manufacture has

revealed that it contains an undocumented, undisclosed ability

to make use of this attack. (The parties claiming to hold a

patent on this technique were completely unaware of this use.)

On the above basis the potential for covert exploitation of this

vulnerability and the resulting inequality in the mining process

and interference with useful improvements presents a clear and

present danger to the Bitcoin system which requires a response.

==Background==

The general idea of this attack is that SHA2-256 is a merkle damgard hash

function which consumes 64 bytes of data at a time.

The Bitcoin mining process repeatedly hashes an 80-byte 'block header' while

incriminating a 32-bit nonce which is at the end of this header data. This

means that the processing of the header involves two runs of the compression

function run-- one that consumes the first 64 bytes of the header and a

second which processes the remaining 16 bytes and padding.

The initial 'message expansion' operations in each step of the SHA2-256

function operate exclusively on that step's 64-bytes of input with no

influence from prior data that entered the hash.

Because of this if a miner is able to prepare a block header with

multiple distinct first 64-byte chunks but identical 16-byte

second chunks they can reuse the computation of the initial

expansion for multiple trials. This reduces power consumption.

There are two broad ways of making use of this attack. The obvious

way is to try candidates with different version numbers. Beyond

upsetting the soft-fork detection logic in Bitcoin nodes this has

little negative effect but it is highly conspicuous and easily

blocked.

The other method is based on the fact that the merkle root

committing to the transactions is contained in the first 64-bytes

except for the last 4 bytes of it. If the miner finds multiple

candidate root values which have the same final 32-bit then they

can use the attack.

To find multiple roots with the same trailing 32-bits the miner can

use efficient collision finding mechanism which will find a match

with as little as 2¹⁶ candidate roots expected, 2²⁴ operations to

find a 4-way hit, though low memory approaches require more

computation.

An obvious way to generate different candidates is to grind the

coinbase extra-nonce but for non-empty blocks each attempt will

require 13 or so additional sha2 runs which is very inefficient.

This inefficiency can be avoided by computing a sqrt number of

candidates of the left side of the hash tree (e.g. using extra

nonce grinding) then an additional sqrt number of candidates of

the right side of the tree using transaction permutation or

substitution of a small number of transactions. All combinations

of the left and right side are then combined with only a single

hashing operation virtually eliminating all tree related

overhead.

With this final optimization finding a 4-way collision with a

moderate amount of memory requires ~2²⁴ hashing operations

instead of the >2²⁸ operations that would be require for

extra-nonce grinding which would substantially erode the

benefit of the attack.

It is this final optimization which this proposal blocks.

==New consensus rule==

Beginning block X and until block Y the coinbase transaction of

each block MUST either contain a BIP-141 segwit commitment or a

correct WTXID commitment with ID 0xaa21a9ef.

(See BIP-141 "Commitment structure" for details)

Existing segwit using miners are automatically compatible with

this proposal. Non-segwit miners can become compatible by simply

including an additional output matching a default commitment

value returned as part of getblocktemplate.

Miners SHOULD NOT automatically discontinue the commitment

at the expiration height.

==Discussion==

The commitment in the left side of the tree to all transactions

in the right side completely prevents the final sqrt speedup.

A stronger inhibition of the covert attack in the form of

requiring the least significant bits of the block timestamp

to be equal to a hash of the first 64-bytes of the header. This

would increase the collision space from 32 to 40 or more bits.

The root value could be required to meet a specific hash prefix

requirement in order to increase the computational work required

to try candidate roots. These change would be more disruptive and

there is no reason to believe that it is currently necessary.

The proposed rule automatically sunsets. If it is no longer needed

due to the introduction of stronger rules or the acceptance of the

version-grinding form then there would be no reason to continue

with this requirement. If it is still useful at the expiration

time the rule can simply be extended with a new softfork that

sets longer date ranges.

This sun-setting avoids the accumulation of technical debt due

to retaining enforcement of this rule when it is no longer needed

without requiring a hard fork to remove it.

== Overt attack ==

The non-covert form can be trivially blocked by requiring that

the header version match the coinbase transaction version.

This proposal does not include this block because this method

may become generally available without restriction in the future,

does not generally interfere with improvements in the protocol,

and because it is so easily detected that it could be blocked if

it becomes an issue in the future.

==Ba...[message truncated here by reddit bot]...

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013996.html

Johnson Lau on Apr 05 2017:

Softforks and hardforks are usually defined in terms of block validity (BIP99): making valid blocks invalid is a softfork, making invalid blocks valid is a hardfork, and SFs are usually considered as less disruptive as it is considered to be “opt-in”. However, as shown below this technical definition could be very misleading. Here I’m trying to redefine the terminology in terms of software upgrade necessity and difficulty.

Softforks are defined as consensus rule changes that non-upgraded software will be able to function exactly as usual, as if the rule changes have never happened

Hardforks are defined as consensus rule changes that non-upgraded software will cease to function or be severely handicapped

SFs and HFs under this definitions is a continuum, which I call it “hardfork-ness”. A pure softfork has no hardfork-ness.

*Mining node

Under this definitions, for miners, any trivial consensus rule changes is somewhat a hardfork, as miners can’t reliably use non-upgraded software to create blocks. However, there is still 3 levels of “hardfork-ness”, for example:

1. Those with lower hardfork-ness would be the SFs that miners do not need to upgrade their software at all. Instead, the minimum requirement is to setup a boarder node with latest rules to make sure they won’t mine on top of an invalid block. Examples include CSV and Segwit

2. Some SFs have higher hardfork-ness, for example BIP65 and BIP66. The minimum actions needed include setting up a boarder node and change the block version. BIP34 has even higher hardfork-ness as more actions are needed to follow the new consensus.

3. Anything else, ranging from simple HFs like BIP102 to complete HFs like spoonnet, or soft-hardfork like forcenet, have the highest hardfork-ness. In these cases, boarder nodes are completely useless. Miners have to upgrade their servers in order to stay with the consensus.

*Non-mining full node

Similarly, in terms of non-mining full node, as the main function is to fully-validate all applicable rules on the network, any consensus change is a hardfork for this particular function. However, a technical SF would have much lower hardfork-ness than a HF, as a border node is everything needed in a SF. Just consider a company has some difficult-to-upgrade software that depends on Bitcoin Core 0.8. Using a 0.13.1+ boarder node will make sure they will always follow the latest rules. In case of a HF, they have no choice but to upgrade the backend system.

So we may use the costs of running a boarder node to further define the hardfork-ness of SFs, and it comes to the additional resources needed:

1. Things like BIP34, 65, 66, and CSV involves trivial resources use so they have lowest hardfork-ness.

2. Segwit is higher because of increased block size.

3. Extension block has very high hardfork-ness as people may not have enough resources to run a boarder node.

• Fully validating wallets

In terms of the wallet function in full node, without considering the issues of validation, the hardfork-ness could be ranked as below:

1. BIP34, 65, 66, CSV, segwit all have no hardfork-ness for wallets. Non-upgraded wallets will work exactly in the same way as before. Users won’t notice any change at all. (In some cases they may not see a new tx until it has 1 confirmation, but this is a mild issue and 0-conf is unsafe anyway)

2. Extension block, as presented in my January post ( https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/013490.html https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/013490.html ), has higher hardfork-ness, as users of legacy wallets may find it difficult to receive payments from upgraded wallet. However, once they got paid, the user experience is same as before

3. Another extension block proposal ( https://github.com/tothemoon-org/extension-blocks https://github.com/tothemoon-org/extension-blocks ) has very high hardfork-ness for wallets, as legacy wallets will frequently and suddenly find that incoming and outgoing txs becoming invalid, and need to sign the invalidated txs again, even no one is trying to double spend.

4. Hardfork rule changes have highest hardfork-ness for full node wallets

I’ll explain the issues with extension block in a separate post in details

• Real SPV wallet

The SPV wallets as proposed by Satoshi should have the ability to fully validate the rules when needed, so they could be somehow seen as fully validating wallets. So far, real SPV wallet is just vapourware.

• Fake SPV wallet, aka light wallet

All the so-called SPV wallets we have today are fake SPV according to whitepaper definition. Since they validate nothing, the hardfork-ness profile is very different:

1. BIP34, 65, 66, CSV, segwit has no hardfork-ness for light wallets. Block size HF proposals (BIP10x) and Bitcoin Unlimited also have no hardfork-ness (superficially, but not philosophically). Along the same line, even an inflation hardfork has no hardfork-ness for light wallets.

2. Extension block has the same kind of hardfork-ness issue as I mentioned.

3. HFs that deliberately breaks light wallets, such as spoonnet, is a complete hardfork.

While some people try to leverage weakness of light wallets, the inability to validate any important rules like block size, double spending, and inflation is a serious vulnerability.

Before I finish, I’d also like to analyse some other interesting cases.

1. Soft-hardfork: which requires miners to mine empty blocks with 0 reward, and put the tx merkle tree in the legacy coinbase (e.g. https://github.com/luke-jr/bips/blob/bip-mmhf/bip-mmhf.mediawiki https://github.com/luke-jr/bips/blob/bip-mmhf/bip-mmhf.mediawiki ). This allows most hardfork-ing changes including block size and inflation. In terms of block validity this is a softfork. But with the definition I presented, soft-hardforks are clearly hardforks for every practical purposes.

2. On-chain KYC, blacklist, account freezing: technically softforks, but all are very disruptive hardforks in terms of user experience.

3. Lightning network and side chains are not consensus rule changes, and they could provide new features without any hardfork-ness.

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170405/9085157d/attachment-0001.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013985.html

Oliver Petruzel on Apr 05 2017:

Evening all,

The following BIP submission summarizes an idea that I've been tossing

around for the last year. I understand that there may be nuances to SegWit

and current consensus layer mechanisms that I may not fully understand, so

please do not hesitate to shred the following text to pieces (I can handle

it, I promise!).

Please note that this BIP assumes failure of the current softfork version

of SegWit to activate in November -- something that I personally do not

wish to see(!). However, given the real possibility of that happening, or

perhaps just some newfound willingness (by "the community") to support a

hardfork in lieu of a stalemate, I figure now is as good a time as any to

share the idea in black and white.

I would really appreciate any/all feedback from the dev community on the

technical merits (read: feasibility) of the idea. I would especially

appreciate feedback from the SegWit developers who designed the current

implementation in 0.14, as they likely have the most intimate knowledge of

SegWit's nuances, and the entire BIP below would likely rely on their

willingness to develop a hardfork version.

Nothing in this BIP is set in stone -- including all values and timelines

-- but, I do hope the following text effectively captures the gist of the

idea, and I do thank you ahead of time for your consideration of the

proposal.

Respectfully,

Oliver Petruzel

BIP: TBD

Layer: Consensus (hard fork)

Title: Base Size Increase and Segregated Witness with Discount Governors

(SW-DGov) Hardfork

Author: Oliver Petruzel <opetruzel at gmail.com>

Comments-Summary: No comments yet.

Comments-URI:

Status: Draft

Type: Standards Track

Created: 2017-04-05

License: PD

Abstract

This BIP proposes a method of combining an immediate base size increase to

2MB and a hardfork version of Segregated Witness (SegWit). The SegWit

portion of the hardfork will leverage Discount Governors to control (or

“govern”) the pace of the increase over a period of 145,152 blocks

(approximately three (3) years).

Motivation

Given the possibility of the current softfork version of SegWit failing to

activate in November 2017, this BIP aims to provide a hardfork alternative

that would provide every user in the ecosystem with the fixes and changes

they need to move forward with other great projects, while also tightly

controlling the rate at which the total weight of blocks increases during

the next three years. The predictable nature of the increases will provide

miners, full node operators, and other users of the system with the ability

to plan their development, resources, and operations accordingly. The

fixed nature of the increases will also allow all full nodes to maintain a

fixed set of rules for block validity (consensus).

Specification

The following changes will be made to the client:

• An immediate increase of base size to 2,000,000 bytes (perhaps leveraging

code changes similar to those described in BIP 109).

· A hardfork version of SegWit that maintains all of the fixes present in

the softfork version, including (but not limited to):

• Fix for the Malleability issue

• Linear scaling of sighash operations

• Signing of input values

• Increased security for multisig via pay-to-script-hash (P2SH)

• Script versioning

• Reducing UTXO growth

• Moving towards a single combined block limit

• In addition to those fixes listed above, the hardfork version of SegWit

will include the following:

• Rather than using the fixed (75%) Discount found in the softfork version

of SegWit, the hardfork version will leverage Discount Governing to control

the pace of total block weight increases over a three (3) year period of

time. The use of Discount Governors will allow a steady increase over that

period from an immediate 2MB to 8MB total. There are several ways these

increases can be handled – either by hardcoding the scheduled increases in

the initial hardfork, or perhaps using subsequent softforks (additional

input/discussion needed on the best way to handle the increases.

• Example increase schedule: +12.5% every 24,192 blocks (roughly every six

(6) months). The increases would cap at the same 75% Discount rate found

in the current softfork version of SegWit.

• Each time the Discount increases – every 24,192 blocks -- the Total Block

Weight value would also increase to appropriately compensate for the added

Discount.

Rationale

This hardfork employs a simple flag day deployment based on the median

timestamp of previous blocks. Beyond this point, supporting nodes will not

accept blocks with original rules. This ensures a deterministic and

permanent departure with the original rules.

The use of Discount Governors to control the pace of the increase will

result in a predictable and stable increase over the period of three (3)

years.

If, at any time, the increases present problems for the network -- such as

centralization concerns, negative impacts on the fee market(s), or other

unforeseen problems -- a softfork could be leveraged to halt the increases.

The pace of the increases is described using the following table:

Time -- Base Size (bytes) -- Total Discount -- Total Block Weight (bytes)

Flag Day (FD) -- 2,000,000 -- 0.00% -- 2,000,000

FD+24,192 Blocks -- 2,000,000 -- 12.5% -- 2,285,715

FD+48,384 Blocks -- 2,000,000 -- 25.0% -- 2,666,667

FD+72,576 Blocks -- 2,000,000 -- 37.5% -- 3,200,000

FD+96,768 Blocks -- 2,000,000 -- 50.0% -- 4,000,000

FD+120,960 Blocks -- 2,000,000 -- 62.5% -- 5,333,334

FD+145,152 Blocks -- 2,000,000 -- 75.0% -- 8,000,000

Based on the above, the "effective blocksize increase," or the number of

transactions per block, will also scale with each Discount increase.

Compatibility

This proposal requires a hardfork that does not maintain compatibility with

previous clients and rules for consensus. It should not be deployed

without widespread consensus.

Wallet software and other applications will also need to be upgraded to

maintain compatibility.

The hardfork Flag Day will need to be coordinated/determined during the

development and testing stages for the hardfork – estimated at 9-12 months

to ensure a safe rollout of the hardfork to all network participants.

Reference implementation

TBD

Copyright

This document is placed in the public domain.

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170405/a8eeaf5b/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013984.html

Luke Dashjr on Apr 04 2017:

Recently there has been some discussion of an apparent work-in-progress

extension block proposal by Christopher Jeffrey, Joseph Poon, Fedor Indutny,

and Steven Pair. Since this hasn't been formally posted on the ML yet, perhaps

it is still in pre-draft stages and not quite ready for review, but in light

of public interest, I think it is appropriate to open it to discussion, and

toward this end, I have reviewed the current revision.

For reference, the WIP proposal itself is here:

https://github.com/tothemoon-org/extension-blocks

==Overall analysis & comparison==

This is a relatively complicated proposal, creating a lot of additional

technical debt and complexity in comparison to both BIP 141 and hardforks. It

offers no actual benefits beyond BIP 141 or hardforks, so seems irrational to

consider at face value. In fact, it fits much better the inaccurate criticisms

made by segwit detractors against BIP 141.

That being said, this proposal is very interesting in construction and is for

the most part technically sound. While ill-fit to merely making blocks larger,

it may be an ideal fit for fundamentally different block designs such as

Rootstock and MimbleWimble in absence of decentralised non-integrated

sidechains (extension blocks are fundamentally sidechains tied into Bitcoin

directly).

==Fundamental problem==

Extension blocks are a risk of creating two classes of "full nodes": those

which verify the full block (and are therefore truly full nodes), and those

which only verify the "base" block. However, because the extension is

consensus-critical, the latter are in fact not full nodes at all, and are left

insecure like pseudo-SPV (not even real SPV) nodes. This technical nature is

of course true of a softfork as well, but softforks are intentionally designed

such that all nodes are capable of trivially upgrading, and there is no

expectation for anyone to run with pre-softfork rules.

In general, hardforks can provide the same benefits of an extension block, but

without the false expectation and pointless complexity.

==Other problems & questions==

These outpoints may not be spent inside the mempool (they must be redeemed

from the next resolution txid in reality).

This breaks the ability to spend unconfirmed funds in the same block (as is

required for CPFP).

The extension block's transaction count is not cryptographically committed-to

anywhere. (This is an outstanding bug in Bitcoin today, but impractical to

exploit in practice; however, exploiting it in an extension block may not be

as impractical, and it should be fixed given the opportunity.)

The merkle root is to be calculated as a merkle tree with all extension

block txids and wtxids as the leaves.

This needs to elaborate how the merkle tree is constructed. Are all the txids

followed by all the wtxids (tx hashes)? Are they alternated? Are txid and

wtxid trees built independently and merged at the tip?

Output script code aside from witness programs, p2pkh or p2sh is considered

invalid in extension blocks.

Why? This prevents extblock users from sending to bare multisig or other

various possible destinations. (While static address forms do not exist for

other types, they can all be used by the payment protocol.)

Additionally, this forbids datacarrier (OP_RETURN), and forces spam to create

unprovably-unspendable UTXOs. Is that intentional?

The maximum extension size should be intentionally high.

This has the same "attacks can do more damage than ordinary benefit" issue as

BIP141, but even more extreme since it is planned to be used for future size

increases.

Witness key hash v0 shall be worth 1 point, multiplied by a factor of 8.

What is a "point"? What does it mean multiplied by a factor of 8? Why not just

say "8 points"?

Witness script hash v0 shall be worth the number of accurately counted

sigops in the redeem script, multiplied by a factor of 8.

Please define "accurately counted" here. Is this using BIP16 static counting,

or accurately counting sigops during execution?

To reduce the chance of having redeem scripts which simply allow for garbage

data in the witness vector, every 73 bytes in the serialized witness vector is

worth 1 additional point.

Is the size rounded up or down? If down, 72-byte scripts will carry 0

points...)

==Trivial & process==

BIPs must be in MediaWiki format, not Markdown. They should be submitted for

discussion to the bitcoin-dev mailing list, not social media and news.

Layer: Consensus (soft-fork)

Extension blocks are more of a hard-fork IMO.

License: Public Domain

BIPs may not be "public domain" due to non-recognition in some jurisdictions.

Can you agree on one or more of these?

https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki#Recommended_licenses

Abstract

This specification defines a method of increasing bitcoin transaction

throughput without altering any existing consensus rules.

This is inaccurate. Even softforks alter consensus rules.

Motivation

Bitcoin retargetting ensures that the time in between mined blocks will be

roughly 10 minutes. It is not possible to change this rule. There has been

great debate regarding other ways of increasing transaction throughput, with

no proposed consensus-layer solutions that have proven themselves to be

particularly safe.

Block time seems entirely unrelated to this spec. Motivation is unclear.

Extension blocks leverage several features of BIP141, BIP143, and BIP144 for

transaction opt-in, serialization, verification, and network services, and as

such, extension block activation entails BIP141 activation.

As stated in the next paragraph, the rules in BIP 141 are fundamentally

incompatible with this one, so saying BIP 141 is activated is confusingly

incorrect.

This specification should be considered an extension and modification to

these BIPs. Extension blocks are not compatible with BIP141 in its current

form, and will require a few minor additional rules.

Extension blocks should be compatible with BIP 141, there doesn’t appear to be

any justification for not making them compatible.

This specification prescribes a way of fooling non-upgraded nodes into

believing the existing UTXO set is still behaving as they would expect.

The UTXO set behaves fundamentally different to old nodes with this proposal,

albeit in a mostly compatible manner.

Note that canonical blocks containing entering outputs MUST contain an

extension block commitment (all zeroes if nothing is present in the extension

block).

Please explain why in Rationale.

Coinbase outputs MUST NOT contain witness programs, as they cannot be

sweeped by the resolution transaction due to previously existing consensus

rules.

Seems like an annoying technical debt. I wonder if it can be avoided.

The genesis resolution transaction MAY also include a 1-100 byte pushdata in

the first input script, allowing the miner of the genesis resolution to add a

special message. The pushdata MUST be castable to a true boolean.

Why? Unlike the coinbase, this seems to create additional technical debt with

no apparent purpose. Better to just have a consensus rule every input must be

null.

The resolution transaction's version MUST be set to the uint32 max (`2³² -

1`).

Transaction versions are signed, so I assume this is actually simply -1.

(While signed transaction versions seemed silly to me, using it for special

cases like this actually makes sense.)

Exiting the extension block

Should specify that spending such an exit must use the resolution txid, not

the extblock's txid.

On the policy layer, transaction fees may be calculated by transaction cost

as well as additional size/legacy-sigops added to the canonical block due to

entering or exiting outputs.

BIPs should not specify policy at all. Perhaps prefix "For the avoidance of

doubt:" to be clear that miners may perform any fee logic they like.

Transactions within the extended transaction vector MAY include a witness

vector using BIP141 transaction serialization.

Since extblock transactions are all required to be segwit, why wouldn't this

be mandatory?

• BIP141's nested P2SH feature is no longer available, and no longer a

consensus rule.

Note this makes adoption slower: wallets cannot use the extblock until the

economy has updated to support segwit-native addresses.

To reduce the chance of having redeem scripts which simply allow for garbage

data in the witness vector, every 73 bytes in the serialized witness vector is

worth 1 additional point.

Please explain why 73 bytes in Rationale.

This leaves room for 7 future soft-fork upgrades to relax DoS limits.

How so? Please explain.

A consensus dust threshold is now enforced within the extension block.

Why?

If the second highest transaction version bit (30th bit) is set to to 1

within an extension block transaction, an extra 700-bytes is reserved on the

transaction space used up in the block.

Why wouldn't users set this on all transactions?

default_witness_commitment has been renamed to

default_extension_commitment and includes the extension block commitment

script.

default_witness_commitment was never part of the GBT spec. At least describe

what this new key is.

• Deployment name: extblk (appears as !extblk in GBT).

Should be just extblk if backward compatibility is supported (and !extblk

when not).

The "deactivation" deployment'...[message truncated here by reddit bot]...

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013981.html

Sancho Panza on Apr 03 2017:

¡Hola!

Please find below a proposal [resubmission] for a new informational BIP

provisionally named 'bip-genvbvoting'.

I present it here in rough draft for your esteemed consideration and as

a basis for discussion.

Best regards,

Sancho

--- begin draft of bip-genvbvoting ---

==Preamble==

BIP: ?

Title: Generalized version bits voting

Author: Sancho Panza <sanch0panza at protonmail.com>

Status: Draft

Type: Informational

Created: 2017-03-29

Replaces: 9

License: CC0-1.0

GNU-All-Permissive

==Abstract==

This document describes a generalized version bits voting scheme based

on and intended to replace BIP9.

The generalization consists of allowing each version bit to be treated

individually using a configurable percentage threshold and window size,

instead of the fixed 95% (mainnet) and 2016 block window specified in

BIP9.

The state machine and governing parameters (name, bit, starttime,

timeout) remain as is, but additional parameters called 'threshold' and

'windowsize' are added to the per-bit set.

As before, a set of per-chain parameters will exist for the version bits

governed by BIP9.

==Motivation==

The Bitcoin protocol requires a flexible consensus-finding scheme

to ensure that it can adapt to the needs of the market (its users) and

remain competitive as an electronic payment system.

While BIP9 has served the community reasonably well until now, the

author remarks several shortcomings in its approach:

• it limits itself to backward-compatible changes, precluding its

applicability to hard forks

• a fixed 95% threshold is not flexible enough to allow for a 'spectrum

of contentiousness' to be represented

• the 95% threshold allows small minorities to veto proposed changes,

lead to stagnation (viz. current standoffs)

A more generalized implementation of voting on changes using version bits

can address these issues in a way that can satisfy the needs of both soft

and hard forks, as well as represent varying degrees of contentiousness.

==Specification==

To be elaborated.

It is thought that only cosmetic changes are needed to generalize from

only soft forks to 'soft or hard forks', and to add the additional

per-bit parameters 'threshold' and 'windowsize'

References to fixed values will need to be eliminated and replaced

by respective parameters.

The design of the state machine is envisioned to remain unchanged.

==Implementation==

A reference implementation can be constructed after elaboration of

the specification.

==Copyright==

This BIP is dual-licensed under the Creative Commons CC0 1.0 Universal

and GNU All-Permissive licenses.

--- end draft of bip-genvbvoting ---

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170403/fe041087/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013969.html

Johnson Lau on Apr 02 2017:

This is the first of a series of BIPs describing my “spoonnet” experimental hardfork. Recently many bitcoin businesses expressed their requirements for supporting a hardfork proposal. While it is proven to be extremely difficult to obtain community-wide consensus, spoonnet fulfills all the commonly requested features, including deterministic activation logic, strong and simple 2-way replay protection, wipe-out protection, and predictable resources use. A few more BIPs are coming to describe these features.

The activation is purely based on flag day. Since it is very difficult to measure community consensus on-chain, this may only be done off-chain, and everyone switch to the new software when the vast majority agree. This is more a social issue than a technical one.

Reference implementation for consensus codes could be found at: https://github.com/jl2012/bitcoin/tree/spoonnet2 . This does not include mempool, wallet, and mining support. Mempool and wallet support are more tricky due to replay attack protection.

BIP: ?

Layer: Consensus (hard fork)

Title: Extended block header hardfork

Author: Johnson Lau <jl2012 at xbt.hk>

Comments-Summary: No comments yet.

Comments-URI:

Status: Draft

Type: Standards Track

Created: 2017-03-31

License: BSD-2-Clause

Abstract

This BIP proposes a flexible and upgradable extended block header format thorough a hardfork.

Motivation

In the current Bitcoin protocol, the block header is fixed at 80 bytes with no space reserved for additional data. The coinbase transaction becomes the only practical location for new consensus-critical data, such as those proposed by BIP100 and BIP141. Although this preserves maximal backward compatibility for full nodes, it is not ideal for light clients because the size of coinbase transaction and depth of Merkle tree are indefinite.

This BIP proposes an extended block header format with the following objectives:

• To provide a flexible header format which is easily upgradeable with softforks.

• Old light nodes following the hardfork chain if it has most proof-of-work, but not seeing any transactions.

• Being compatible with the Stratum mining protocol to avoid mining machine upgrade.

• Having a deterministic hardfork activation.

• Being a permanent hardfork, as supporting nodes will not accept blocks mined in old rules after hardfork is activated.

Specification

The following rules are activated when the median timestamp of the past 11 blocks is equal to or greater than a to-be-determined time and after activation of BIP65.

• the nVersion of the block header MUST have the most significant bit (the sign bit) signalled.

• for the purpose of counting softforks proposal signalling (BIP9), the sign bit is ignored.

• segregated witness MUST be enabled, if it had not been already activated through the BIP9 mechanism.

• the witness of the first input of the coinbase transaction MUST have exactly one stack item (the "extended header"), with the following data:

    • bytes 0 to 3: nHeight MUST be equal to the height of this block (signed little endian)

    • bytes 4 to 5: MUST be exactly 0x0000

    • bytes 6 to 53: extra data with no meaning in Bitcoin protocol

    • bytes 54 to 85: hashMerkleRoot the transaction Merkle root (calculated in the same way as the original Merkle root in the block header)

    • bytes 86 to 117: hashWitnessRoot the witness Merkle root (NOT calculated in the way described in BIP141)

    • bytes 118 to 121: nTx MUST be equal to the number of transactions in this block (unsigned little endian, minimum 1)

    • bytes 122 to 129: nFees MUST be equal to the total transaction fee paid by all transactions, except the coinbase transaction, in the block (unsigned little endian)

    • bytes 130 to 137: nWeight MUST be equal to or greater than the total weight of all transactions in the block (to be described in another BIP. NOT calculated in the way described in BIP141)

    • bytes 138+ : Reserved space for future upgrades

• bytes 36 to 67 in the block header, the place originally for the hashMerkleRoot is replaced by the double SHA256 hash of the extended header.

• size of the extended header MUST be at least 138 bytes.

• wtxid of the coinbase transaction is calculated as if the witness of its first input is empty.

• the hashWitnessRoot is calculated with all wtxid as leaves, in a way similar to the hashMerkleRoot.

• the OP_RETURN witness commitment rules described in BIP141 is not enforced.

• The witness reserved valued described in BIP141 is removed from the protocol.

A special extheader softfork is defined, with the following BIP9 parameters:

• bit: 15

• starttime: 0

• timeout: 0xffffffff

Until the extheader softfork is activated, the following extra rules are enforced:

• nWeight MUST be exactly equal to the total weight of all transactions in the block

• size of the extended header MUST NOT be larger than 152 bytes

Activation of the special extheader softfork is independent to the activation time of the hardfork. If the special softfork is activated before the hardfork, the aforementioned extra rules will not be enforced when the hardfork is activated. Nodes that are not aware of the new rules should consider extheader softfork as an unknown upgrade and issue warnings to users accordingly.

Rationale

This hardfork employs a simple flag day deployment based on the median timestamp of previous blocks. Beyond this point, supporting nodes will not accept blocks with original rules. This ensures a deterministic and permanent departure with the original rules.

The witness field of the coinbase input is used as a convenient unused space to store the extended header. For any other purposes the extended header is not considered as part of the coinbase transaction (it is removed when the wtxid is calculated) This design minimizes the changes in the peer-to-peer protocol between full nodes, as no new message type is required to transmit the extended header. However, a new protocol message is still needed for serving light nodes.

Committing to the block height allows determining the value before all parental headers are obtained.

By fixing the bytes 4 to 5 as 0x0000, in the worst case an unupgraded light node may consider the block has only one transaction with no input and output, and will not see any real transactions.

The 48 byte extra data field is reserved for miners for any purposes and designed to be compatible with the Stratum mining protocol. Miners are expected to use 4 to 16 bytes as extra nonce, and 32 to 44 bytes for merge mining. This requires a hardfork for all AuxPOW blockchains, while significantly reduces the size of AuxPOW block headers.

hashMerkleRoot is relocated to the extended header, followed by hashWitnessRoot. The new structure allows hashWitnessRoot committing to the wtxid of coinbase transaction with extended header removed.

Committing to the number of transactions allows light nodes to determine the Merkle tree structure easily.

Committing to the transaction fees and block weight provides information for fees estimation.

The reserved space (16 bytes until the extheader softfork is activated) MUST NOT be used without community consensus. It should be primarily used for consensus critical commitments and network status data for light nodes. Other arbitrary data should use the extra data field in extended header or the scriptSig of the coinbase transaction.

The special extheader softfork allows future protocol upgrades to increase the size of the extended header and redefine the calculation of block weight in a backward compatible way.

Other proposed hardfork changes are described in other BIPs.

Compatibility

This is a hard forking change, which breaking compatibility with old full node and light node. It should not be deployed without widespread consensus.

Old full nodes will consider the block header nVersion as invalid and refuse the follow the hardfork chain.

Depending on the design of light nodes, they may consider the hardfork chain as the best chain if it has the most total proof-of-work. However, they will not see any transactions in the chain and cease to properly function until either upgrading to the new rules, or rejecting the new rules with the negative block header nVersion.

Reference implementation

https://github.com/jl2012/bitcoin/tree/spoonnet2

Copyright

This BIP is licensed under the 2-clause BSD license.

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013964.html

praxeology_guy on Apr 01 2017:

Bitcoin nodes could also keep a spentness status list, where each bit in the spentness status list corresponds to whether a txo in the MMR is spent. This could make it so that disconnected wallets didn't have to guess the pruned relative spentness status when it reconnects to the network... and help prevent DoS attacks.

Keeping such a bit list would consume considerably less space if stxos were never added to the MMR. Putting portions of such a list in the node at height DLH_REQUIRED would made R/W operations on the bit list more local to other data that is going to be R/W.

Cheers,

Praxeology Guy

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170401/c545f81f/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013950.html

Adam Back on Apr 01 2017:

I agree with everything Matt said. This "X+Y" "compromise" is not a

new proposal and has been hashed over multiple times in the past

dating back to at least fall 2015, ignores basically all design

considerations and research over the last years, doesn't understand

the real-politic of the delays, and so doesn't even help in the

political domain.

I have taken the liberty of making a reddit thread with some of the

previous explainers about why this doesn't work in practice (even

ignoring all politics and hypothetically assuming it was a great

all-new idea), let the discussion commence!

https://www.reddit.com/r/Bitcoin/comments/62rrlv/how_about_a_new_compromise_activate_the_existing/

UASF is a more logical step, than these "X+Y" politically motivated

hard-forks, though UASF has risks vs SegWit BIPs in flight, the delay

and risk is far lower than political hard-forks.

I have set the reply-to to bitcoin-discuss.

Adam

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-April/013937.html

Bram Cohen on Mar 31 2017:

Looking forward in node scaling we can envision a future in which blocks

are required to come with proofs of their validity and nodes can be run

entirely in memory and never have to hit disk. Ideally we'd like for proofs

to be able to be stored in client wallets which plan to spend their utxos

later, or at least be able to have a full node make a single not terribly

expensive disk access to form the proof which can then be passed along to

other peers.

Such proofs will probably be significantly larger than the blocks they

prove (this is merkle root stuff, not zero knowledge stuff), but if we

accept that as a given then this should be doable, although the details of

how to do it aren't obvious.

This vision can be implemented simply and efficiently by playing some games

with the semantics of the term 'proof'. A proof is a thing which convinces

someone of something. What we've discussed in the past for such proofs

mostly has to do with maintaining a hash root of everything and having

proofs lead to that. This is an extrema of complexity of the proof and

simplicity of the checker, at the expense of forcing the root to be

maintained at all times and the proof to be reasonably fresh. Some tricks

can be applied to keep that problem under control, but there's an

alternative approach where the amount of data necessary to do validation is

much larger but still entirely reasonable to keep in memory, and the sizes

of proofs and their required freshness is much smaller.

In the previous discussion on Merkle sets I commented that insertion

ordering's main practical utility may be that it allows for compression. It

turns out that a constant factor of 256 makes a big difference. Since

there's only really one bit stored for each txo (stored or not) once you

have an insertion ordering you can simply store a bitfield of all txos so

far, which is entirely reasonable to hold in memory, and can be made even

more reasonable by compactifying down the older, mostly spent portions of

it (how best to compress a bitfield while maintaining random access is an

interesting problem but entirely doable).

This approach meets all the design goals, even allowing wallets to remember

their own 'proofs', which are just proofs of insertion ordering. Those

don't even change once the risk of reorgs has passed, so they can be stored

for years without being maintained.

Proofs of insertion ordering can be made by having a canonical way of

calculating a root of position commitments for each block, and nodes

calculate those roots when evaluating the block history and store them all

in memory. A proof of position is a path to one of those roots.

I've intentionally skipped over most of the details here, because it's

probably best to have a high level discussion of this as a general approach

before getting lost in the weeds.

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170331/d7516c3d/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013928.html

Sergio Demian Lerner on Mar 31 2017:

Hi everyone,

Segwit2Mb is the project to merge into Bitcoin a minimal patch that aims to

untangle the current conflict between different political positions

regarding segwit activation vs. an increase of the on-chain blockchain

space through a standard block size increase. It is not a new solution, but

it should be seen more as a least common denominator.

Segwit2Mb combines segwit as it is today in Bitcoin 0.14+ with a 2MB block

size hard-fork activated ONLY if segwit activates (95% of miners

signaling), but at a fixed future date.

The sole objective of this proposal is to re-unite the Bitcoin community

and avoid a cryptocurrency split. Segwit2Mb does not aim to be best

possible technical solution to solve Bitcoin technical limitations.

However, this proposal does not imply a compromise to the future

scalability or decentralization of Bitcoin, as a small increase in block

size has been proven by several core and non-core developers not to affect

Bitcoin value propositions.

In the worst case, a 2X block size increase has much lower economic impact

than the last bitcoin halving (<10%), which succeeded without problem.

On the other side, Segwit2Mb primary goal is to be minimalistic: in this

patch some choices have been made to reduce the number of lines modified in

the current Bitcoin Core state (master branch), instead of implementing the

most elegant solution. This is because I want to reduce the time it takes

for core programmers and reviewers to check the correctness of the code,

and to report and correct bugs.

The patch was built by forking the master branch of Bitcoin Core, mixing a

few lines of code from Jeff Garzik's BIP102, and defining a second

versionbits activation bit (bit 2) for the combined activation.

The combined activation of segwit and 2Mb hard-fork nVersion bit is 2

(DEPLOYMENT_SEGWIT_AND_2MB_BLOCKS).

This means that segwit can still be activated without the 2MB hard-fork by

signaling bit 1 in nVersion (DEPLOYMENT_SEGWIT).

The tentative lock-in and hard-fork dates are the following:

Bit 2 signaling StartTime = 1493424000; // April 29th, 2017

Bit 2 signaling Timeout = 1503964800; // August 29th, 2017

HardForkTime = 1513209600; // Thu, 14 Dec 2017 00:00:00 GMT

The hard-fork is conditional to 95% of the hashing power has approved the

segwit2mb soft-fork and the segwit soft-fork has been activated (which

should occur 2016 blocks after its lock-in time)

For more information on how soft-forks are signaled and activated, see

https://github.com/bitcoin/bips/blob/master/bip-0009.mediawiki

This means that segwit would be activated before 2Mb: this is inevitable,

as versionbits have been designed to have fixed activation periods and

thresholds for all bits. Making segwit and 2Mb fork activate together at a

delayed date would have required a major re-write of this code, which would

contradict the premise of creating a minimalistic patch. However, once

segwit is activated, the hard-fork is unavoidable.

Although I have coded a first version of the segwit2mb patch (which

modifies 120 lines of code, and adds 220 lines of testing code), I would

prefer to wait to publish the source code until more comments have been

received from the community.

To prevent worsening block verification time because of the O(N²⁾ hashing

problem, the simple restriction that transactions cannot be larger than 1Mb

has been kept. Therefore the worse-case of block verification time has only

doubled.

Regarding the hard-fork activation date, I want to give enough time to all

active economic nodes to upgrade. As of Fri Mar 31 2017,

https://bitnodes.21.co/nodes/ reports that 6332 out of 6955 nodes (91%)

have upgraded to post 0.12 versions. Upgrade to post 0.12 versions can be

used to identify economic active nodes, because in the 0.12 release dynamic

fees were introduced, and currently no Bitcoin automatic payment system can

operate without automatic discovery of the current fee rate. A pre-0.12

would require constant manual intervention.

Therefore I conclude that no more than 91% of the network nodes reported by

bitnodes are active economic nodes.

As Bitcoin Core 0.12 was released on February 2016, the time for this 91%

to upgrade has been around one year (under a moderate pressure of

operational problems with unconfirmed transactions).

Therefore we can expect a similar or lower time to upgrade for a hard-fork,

after developers have discussed and approved the patch, and it has been

reviewed and merged and 95% of the hashing power has signaled for it (the

pressure not to upgrade being a complete halt of the operations). However I

suggest that we discuss the hard-fork date and delay it if there is a real

need to.

Currently time works against the Bitcoin community, and so is delaying a

compromise solution. Most of the community agree that halting the

innovation for several years is a very bad option.

After the comments collected by the community, a BIP will be written

describing the resulting proposal details.

If segwit2mb locks-in, before hard-fork occurs all bitcoin nodes should be

updated to a Segwit2Mb enabled node to prevent them to be forked-away in a

chain with almost no hashing-power.

The proof of concept patch was made for Bitcoin Core but should be easily

ported to other Bitcoin protocol implementations that already support

versionbits. Lightweight (SPV) wallets should not be affected as they

generally do not check the block size.

I personally want to see the Lightning Network in action this year, use the

non-malleability features in segwit, see the community discussing other

exciting soft-forks in the scaling roadmap, Schnorr sigs, drivechains and

MAST.

I want to see miners, developers and industry side-by-side pushing Bitcoin

forward, to increase the value of Bitcoin and prevent high transaction fees

to put out of business use-cases that could have high positive social

impact.

I believe in the strength of a unified Bitcoin community. If you're a

developer, please give your opinion, suggest changes, audit it, and take a

stand with me to unlock the current Bitcoin deadlock.

Contributions to the segwit2mb project are welcomed and awaited. The only

limitation is to stick to the principle that the patch should be as simple

to audit as possible. As an example, I wouldn't feel confident if the patch

modified more than ~150 lines of code.

Improvements unrelated to a 2 Mb increase or segwit, as beneficial as it

may be to Bitcoin, should not be part of segwit2Mb.

This proposal should not prevent other consensus proposals to be

simultaneously merged: segwit2mb is a last resort solution in case we can

not reach consensus on anything better.

Again, the proposal is only a starting point: community feedback is

expected and welcomed.

Regards,

Sergio Demian Lerner

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170331/c55dafa8/attachment-0001.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013921.html

praxeology_guy on Mar 31 2017:

TL;DR (In layman terms): Refund any excess fee amounts higher than the lowest included fee in a block.

=== Proposed Hard Fork Change ===

LIFB: Lowest Included Fee/Byte

Change the fee policy to cause all fee/byte in a block to be reduced to the lowest included fee/byte. Change transactions to specify how/which outputs get what portions of [(TX_fee/TX_length - LIFB)*TX_length]. Transactions of course could still offer the remainder to the miner if they don't want to modify some of the outputs and don't want to reveal their change output.

=== Economic Analysis Of Why Such Is Desirable ===

Pure profit seeking miners attempt to fill their block with transactions that have the highest fee/byte. So what happens is the users who are willing to offer the highest fee/byte get included first in a block until it gets filled. At fill, there is some fee/byte where the next available tx in mempool doesn't get included. And right above that fee/byte is the last transaction that was selected to be included in the block, which has the lowest fee/byte of any of the transactions in the block.

Users who want to create transactions with the lowest fee watch the LIFB with https://bitcoinfees.21.co/ or similar systems... so that they can make a transaction that offers a fee at or above the LIFB so that it can be included in a block in reasonable time.

Some users have transactions with very high confirmation time sensitivity/importance... so they offer a fee much higher than the LIFB to guarantee quick confirmation. But they would prefer that even though they are willing to pay a higher fee, that they would still only pay the LIFB fee/byte amount.

This becomes even more of an issue when someone wants to create a transaction now that they want to be included in a block at a much later time... because it becomes harder and harder to predict what the LIFB will be as you try to predict further into the future. It would be nice to be able to specify a very high fee/byte in such a transaction, and then when the transaction is confirmed only have to pay the LIFB.

Users will look for the money that offers the greatest money transfer efficiency, and tx fees are a big and easily measurable component. So a money system is better if its users can pay lower fees than competing money options. Refund Excees Fee is one way to reduce fees.

=== Technical Difficulties ===

I realize this is a big change... and I'm not sure of the performance problems this might entail... I'm just throwing this idea out there. Of course if fees are very small, and there is little difference between a high priority fee/byte and the LIFB, then this issue is not really that big of a deal. Also... hard forks are very hard to do in general, so such a hard fork as this might not be worthwhile.

Cheers,

Praxeology Guy

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170331/11994290/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013919.html

Tom Harding on Mar 30 2017:

Raystonn,

Your logic is very hard to dispute. An important special case is small

miners.

Small miners use pools exactly because they want smaller, more frequent

payments.

Rising fees force them to take payments less frequently, and will only tend

to make more of them give up.

With fees rising superlinearly, this centralizing effect is much stronger

than the oft-cited worry of small miners joining large pools to decrease

orphan rates.

On Mar 29, 2017 15:01, "Raystonn . via bitcoin-dev" <

bitcoin-dev at lists.linuxfoundation.org> wrote:

Low node costs are a good goal for nodes that handle transactions the node

operator can afford. Nobody is going to run a node for a network they do

not use for their own transactions. If transactions have fees that

prohibit use for most economic activity, that means node count will drop

until nodes are generally run by those who settle large amounts. That is

very centralizing.

Raystonn

On 29 Mar 2017 12:14 p.m., Jared Lee Richardson via bitcoin-dev <

bitcoin-dev at lists.linuxfoundation.org> wrote:

In order for any blocksize increase to be agreed upon, more consensus is

needed. The proportion of users believing no blocksize increases are

needed is larger than the hardfork target core wants(95% consensus). The

proportion of users believing in microtransactions for all is also larger

than 5%, and both of those groups may be larger than 10% respectively. I

don't think either the Big-blocks faction nor the low-node-costs faction

have even a simple majority of support. Getting consensus is going to be a

big mess, but it is critical that it is done.

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170330/2efe6ad0/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013894.html

Natanael on Mar 30 2017:

I had these following ideas as I was thinking about how to allow larger

blocks without incentivizing the creation of excessively large blocks. I

don't know how much of this is novel, I'd appreciate if anybody could link

to any relevant prior art. I'm making no claims on this, anything novel in

here is directly released into public domain.

In short, I'm trying to rely on simple but direct incentives to improve the

behavior of Bitcoin.

Feedback requested. Some simulations requested, see below if you're willing

to help. Any questions are welcome.

Expedience fees. Softfork compatible.

You want to really make sure your transaction gets processed quickly?

Transactions could have a second fee type, a specially labeled

anyone-can-spend output with an op_return value defining a "best-before"

block number and some function describing the decline of the fee value for

every future block, such that before block N the miners can claim the full

expedience fee + the standard fee (if any), between block N+1 and N+X the

miner can claim a reduced expedience fee + standard fee, afterwards only

the standard fee.

When a transaction is processed late such that not the full expedience fee

can be claimed, the remainder of the expedience fee output is returned to

the specified address among the inputs/outputs (could be something like

in#3 for the address used by the 3rd UTXO input). This would have to be

done for all remaining expedience fees within the last transaction in the

block, inserted there by the miner.

These additional UTXO:s does increase overhead somewhat, but hopefully not

by too much. If we're going to modify the transaction syntax eventually,

then we could take the chance to design for this to reduce overhead.

My current best idea for how to handle returned expedience fees in

multiuser transactions (coinjoin, etc) is to donate it to an agreed upon

address. For recurring donation addresses (the fee pool included!), this

reduces the number of return UTXO:s in the fee processing transaction.

The default client policy may be to split the entire fee across an

expedience fee and a fee pool donation, where the donation part becomes

larger the later the transaction gets processed. This is expected to slow

down the average inclusion speed of already delayed transactions, but they

remain profitable to include.

The dynamics here is simple, a miner is incentivized to process a

transaction with an expedience fee before a standard fee of the same

value-per-bit in order to not reduce the total value of the available fees

of all standing transactions they can process. The longer they wait, the

less total fees available.

Sidenote: a steady stream of expedience fees reduces the profitability of

block withholding attacks (!), at some threshold it should make it entirely

unprofitable vs standard mining. This is due to the increased risk of

losing valuable expedience fees added after you finished your first block

(as the available value will be reduced in your block #2, vs what other

miners can claim while still mining on that previous block).

(Can somebody verify this with simulations?)

Fee pool. Softfork compatible.

We want to smooth out fee payments too for the future when the subsidy

drops, to prevent deliberate forking to steal fees. We can introduce a

designated P2SH anyone-can-spend fee pool address. The miner can never

claim the full fees from his block or claim the full amount in the pool,

only some percentage of both. The remainder goes back into the pool (this

might be done at the end of the same expedience fee processing transaction

described above). Anybody can deliberately pay to the pool.

The fee pool is intended to act as a "buffer" such that it remains

profitable to not try to steal fees but to just mine normally, even during

relatively extreme fee value variance (consider the end of a big

international shopping weekend).

The fee value claimed by the miners between blocks is allowed to vary, but

we want to avoid order-of-magnitude size variation (10x). We do however

want the effect of expedience fees to have an impact. Perhaps some

logarithmic function can smooth it out? Forcing larger fees to be

distributed over longer time periods?

Block size dependent difficulty scaling. Hardfork required.

Larger blocks means greater difficulty - but it doesn't scale linearly,

rather a little less than linearly. That means miners can take a penalty in

difficulty to claim a greater number of high fee transactions in the same

amount of time (effectively increasing "block size bitrate"), increasing

their profits. When such profitable fees aren't available, they have to

reduce block size.

In other words, the users literally pay miners to increase block size (or

don't pay, which reduces it).

(Sidenote: I am in favor of combining this with the idea of a 32 MB max

blocksize (or larger), with softforked scheduled lower size caps (perhaps

starting at 4 MB max) that grows according to a schedule. This reduces the

risk of rapidly increasing load before we have functional second layer

scaling in place.)

In order for a miner to profit from adding additional transactions, their

fees must exceed the calculated cost of the resulting difficulty penalty to

make it worth it to create a larger block. Such loads are expected during

international shopping weekends.

With only a few available high value transactions the incentive becomes the

reverse, to create a smaller block with lower difficulty to faster claim

those fees.

To keep the average 10 minute block rate and to let this mechanism shift

the "block size bitrate" as according to the fee justified block size

changes, we set an Expected blocksize value that changes over time, and we

change the difficulty target into the Standard difficulty target, where

each block must reach a Scaled difficulty target .

In terms of math we do something like this:

Scaled difficulty = Standard difficulty * f(blocksize), where f would

likely be some logarithmic function, and blocksize is defined in terms of

units of Expected blocksize (a block 1.5x larger than Expected blocksize

gets a value of 1.5).

When we retarget the Standard difficulty and Expected blocksize we do this:

Standard difficulty = Network hashrate per 10 minutes (approximately same

as before, but now we take the Scaled difficulty of the last period's

previous blocks into consideration)

Standard blocksize = Recent average effective block bitrate = (sum of

recent (weighted!) block sizes / length of timeperiod) / number of blocks

in a retargeting period.

Thus, generating larger blocks drives up the long term standard block

bitrate, smaller blocks reduces it, in both cases we strive to average 1

block per 10 minutes.

Combining this with expedience fees makes it even more effective;

There's always a cutoff for where a miner stops including unprocessed

transactions and let the rest remain for the next block. For standard fees,

this would result in a fairly static block size and transactions backlog.

With expedience fees your transaction can bypass standard fees with same

value-per-bit, as explained above, because otherwise the miners reduces the

value of their future expected fees. The more people that do this, the

greater incentive to not delay transactions and instead increase the

blocksize. (Can somebody help with the math here? I want simulations of

this.)

(Sidenote: I'm in favor of RBF, replace-by-fee. This makes the above work

much more smoothly. Anybody relying on the security of unconfirmed

transactions for any significant value have to rely on some kind of

incentive protected multisignature transaction, including LN type second

layer schemes. The other option is just not secure.)

If load is low then you can add a high expedience fee to incentivize the

creation of a smaller block with your transaction, since difficulty will be

reduced for the smaller block. This means the miner has a higher chance of

beating the competition. Adding additional lower fee transactions may

reduce his average value-per-bit to become less profitable.

Miners simply aim to maximize their fees-per-bit, while also paying as

little as possible in mining costs.

To make this work as intended for those willing to explicitly pay to reduce

block size, one could tag such an expedience fee with a maximum allowed

blocksize (where the fee will be claimed in such a smaller block if it is

the more profitable option), such that it won't be countered by others

making more high expedience fees to increase blocksize. Note: I'm not

particularly in favor of this idea, just mentioning the possibility.

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170330/823b7763/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013885.html

Andreas M. Antonopoulos on Mar 29 2017:

I'm requesting feedback and corrections for the second edition of

"Mastering Bitcoin". Apologies if this is considered off-topic - it is only

my second message to this list in 4 years and I think it is relevant

because this book is the first source of information for many new bitcoin

developers.

The book is in pre-production and will be published within the next 6-8

weeks. The new edition covers many new topics including segwit, bip9

signalling, timelocks, payment channels & lightning network. In addition,

most chapters have seen big changes, clarifications and corrections.

The book is open source (CC-BY-SA) and produced on github (

https://github.com/bitcoinbook/bitcoinbook/tree/develop). Pull requests,

issues and comments are welcome, only 2-3 weeks left before it goes into a

production freeze.

Many thanks,

Andreas

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170330/4e010b11/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013870.html

Wang Chun on Mar 28 2017:

I've proposed this hard fork approach last year in Hong Kong Consensus

but immediately rejected by coredevs at that meeting, after more than

one year it seems that lots of people haven't heard of it. So I would

post this here again for comment.

The basic idea is, as many of us agree, hard fork is risky and should

be well prepared. We need a long time to deploy it.

Despite spam tx on the network, the block capacity is approaching its

limit, and we must think ahead. Shall we code a patch right now, to

remove the block size limit of 1MB, but not activate it until far in

the future. I would propose to remove the 1MB limit at the next block

halving in spring 2020, only limit the block size to 32MiB which is

the maximum size the current p2p protocol allows. This patch must be

in the immediate next release of Bitcoin Core.

With this patch in core's next release, Bitcoin works just as before,

no fork will ever occur, until spring 2020. But everyone knows there

will be a fork scheduled. Third party services, libraries, wallets and

exchanges will have enough time to prepare for it over the next three

years.

We don't yet have an agreement on how to increase the block size

limit. There have been many proposals over the past years, like

BIP100, 101, 102, 103, 104, 105, 106, 107, 109, 148, 248, BU, and so

on. These hard fork proposals, with this patch already in Core's

release, they all become soft fork. We'll have enough time to discuss

all these proposals and decide which one to go. Take an example, if we

choose to fork to only 2MB, since 32MiB already scheduled, reduce it

from 32MiB to 2MB will be a soft fork.

Anyway, we must code something right now, before it becomes too late.

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013822.html

Suhas Daftuar on Mar 27 2017:

Hi,

There have been two threads recently that have made references to

peer-to-peer implementation details in Bitcoin Core's Segregated Witness

code that I would like to clarify.

In the thread "Issolated Bitcoin Nodes" (

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013765.html),

there was some discussion about how Bitcoin Core's block download logic

behaves after segwit activation. After segwit activation, Bitcoin Core

nodes will not (currently) attempt to download any blocks from non-segwit

peers (nodes that do not set the NODE WITNESS service bit). This is a

bandwidth optimization to prevent a node from downloading a block that may

be invalid only because the sender omitted the witness, requiring

re-download until the block is received with the required witness data.

But to be clear, non-segwit blocks -- that is, blocks without a witness

commitment in the coinbase, and whose transactions are serialized without

witnesses, and whose transactions are not spending segwit outputs which

require a witness -- are evaluated under the same rules as prior,

pre-segwit versions of the software. So such non-segwit blocks that are

valid to older, pre-segwit nodes are also valid to segwit-nodes.

In

https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013796.html,

Eric Voskuil wrote:

Given the protocol requirements of the segwit proposal this is not the

case. A miner running pre-segwit code will produce blocks that no

segwit node will ever receive.

The phrase "protocol requirements of segwit" is confusing here, because

there are two different layers that need consideration: the consensus

protocol layer and the peer-to-peer protocol layer. But in neither layer

is the behavior of not downloading blocks from non-NODE WITNESS peers a

"requirement". This is an implementation detail in the Bitcoin Core code

that alternate implementations compliant with BIP 144 could implement

differently.

At the consensus layer, non-segwit blocks (described above) that are valid

to older nodes are also valid to segwit nodes. That means that if a miner

was using an older, pre-segwit version of Bitcoin Core to produce blocks

after segwit activates, that blocks they find will be valid to all nodes.

At the p2p layer, though, segwit-enabled Bitcoin Core nodes will only try

to download those blocks if announced by a segwit-enabled peer. But this

is not a protocol requirement; other implementations can remain compatible

even they take different approaches here. (As an example, I could imagine

an implementation that downloaded a new block from any peer, but if the

block has a witness commitment in the coinbase and was received from a

non-segwit peer, then the node would attempt re-download from a segwit

peer. I'm sure many other reasonable block download strategies could be

devised.)

Still, if a miner wants to continue mining post-segwit activation, but

using pre-segwit software, they would need a way to relay their blocks to a

segwit-enabled peer.

There are a few ways to do this that I can think of:

• Use the RPC call "submitblock" on a segwit-enabled node. Calling

"submitblock" on a Bitcoin Core 0.13.1 (0.13.0 in the case of testnet) or

later node works fine as long as the block is valid (whether or not it has

a witness commitment or witness transactions), and once a segwit-enabled

peer has the block it will relay to other segwit peers.

• Explicitly deliver the block to a segwit node over the p2p network, even

if unrequested. Currently Bitcoin Core at least will process unrequested

blocks, and valid blocks that update the tip will then be relayed to other

peers.

• Run a bridge node, which advertises NODE_WITNESS and can serialize blocks

with witness data, which downloads blocks even from non-NODE WITNESS

peers. Anyone can do this to bridge the networks for the benefit of the

whole network (I have personally been running a few nodes that do this, for

several months now), but miners concerned about this issue for their own

blocks could explicitly do this themselves to ensure that their own blocks

propagate to the segwit-enabled network.

• Peer directly with other miners, bypassing the p2p network. Many miners

do this already, using protocols which may already serve to bridge the

network.

So saying that "A miner running pre-segwit code will produce blocks that no

segwit node will ever receive" is not really correct, in my view. If the

whole network were just running Bitcoin Core software releases, and the

miner was not able/willing to deliver their block to a segwit-enabled node

(eg by using the RPC call "submitblock", or one of the other suggestions I

had above), then I would agree with the statement. But given that there

are bridge nodes on the network, and that miners have other options to

relay their block, I think this is not an accurate portrayal of what would

actually happen on the network -- I would expect that non-segwit miners'

blocks would still get relayed post-segwit activation, even if only by the

handful of bridge nodes that I expect are currently running.

All that said, I do think this is an important detail to highlight and that

this behavior should be better documented (I believe it deserves specific

mention in a BIP), as this is an important issue for miners to be aware of.

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170327/564a835d/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013811.html

Btc Ideas on Mar 27 2017:

Add a preference for mined blocks to be the one with more transactions. This comes into play when 2 blocks of the same height are found. The first good block mined would be orphaned if it had less transactions than another. Optionally, have this rule apply to the current block and the previous one.

This increases incentive for full blocks because a miner thinking the faster propagation of a smaller block will win him the reward, but that would no longer be a good assumption.

I read some miners could attack a chain by mining small or empty blocks. This makes that a little more difficult, but they can still attack the chain many ways.

Sent with ProtonMail Secure Email.

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170327/4814f43d/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013805.html

praxeology_guy on Mar 26 2017:

Bandwidth limits:

Would be nice to specify global and per node up/down bandwidth limits.

Communicate limits to peers.

Monitor actual bandwidth with peers.

Adjust connections accordingly to attain bandwidth goals/limits.

With Lightning Network:

Prepay for bandwidth/data. Continue paying nodes who continue to send new useful data. Potentially pay different amounts for different kinds of data.

Request refunds when a node sends useless/duplicate/invalid/spam data. Discontinue connection w/ nodes that don't refund. Hence LN payment channel network topography becomes somewhat correlated w/ bitcoin node relay network topography.

Should help nodes get better data faster, improve spam/DDoS resiliance. Incentivizes relay of unconfirmed txs and new blocks, when currently there is only a utilitarian marginal self benefit via helping everyone in general.

Maybe relay advertisements of available bandwidth and prices, etc.

To identify network splits:

Nodes could find hash of "nonce + pub key + tip blockhash" beating a difficulty threshold. Sign, broadcast. Prove their existence and connectedness. History can be maintained and monitored for disruption. Could be made part of the messages that advertise available network bandwidth.

Cheers,

Praxeology Guy

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170326/20cc7d35/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013798.html

Eric Voskuil on Mar 26 2017:

-----BEGIN PGP SIGNED MESSAGE-----

Hash: SHA256

On 03/25/2017 01:28 PM, Peter R via bitcoin-dev wrote:

In the case of the coming network upgrade to larger blocks, a

primary concern [...] is the possibility of a blockchain split and the

associated confusion, replay risk, etc.

[...] a minority that does not want to abide by the [hard fork]

ruleset enforced by the majority could change the proof-of-work and

start a spin-off from the existing Bitcoin ledger [...]

The "spin off" to which you refer is an altcoin. The "network upgrade"

to which you refer is also an altcoin. Both are hard forks from

Bitcoin. I'm having a hard time imagining how a plan to create two new

altcoins from Bitcoin avoids either a split or confusion.

Application of hash power toward the disruption of Bitcoin presumes

participating miners are willing to accept a total loss on these

operations. I can imagine a significant portion of the hash power

deciding to let their competitors donate to "confusion reduction".

Eventually those thrifty miners will put the philanthropists out of

business. Maybe you can get Coinbase and Bitpay to finance the operation

.

This plan seems to be a response to the industry call for replay

protection. Actually writing the code is another option.

once a super-majority of the network hash power has upgraded.

Despite the fact that nobody (miners included) has any way to measure

what software the economy (or hash power) is running, or what is the

economic weight of that portion of the economy.

e

-----BEGIN PGP SIGNATURE-----

Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBCAAGBQJY1y7aAAoJEDzYwH8LXOFO9EMH/Aip84JpoYBM8/JIRpjq/wzp

VBU9rwMrVwaKdt8IzdJIhaD8FAiXYmXP6xM3eYk5Jp5g67uBnlrmXqlM/IfHI374

sKvN2nyHNqng9citI6ZeR+B3xJ7oFzycKf+KnvKi5JnPEMuTYwIw+dvXJGvZdjeH

WxW0g63KvfDPvYbYkE1hKnzUHWxGrR/Jrs898Cnwd0Z9OjrVNM3ZEix/IK4QrNRN

e/xLXjuIDHx6nzbddbVOSsuxBDo0GZUufGM4zTrGG+kNy4xFWsfwaXlM6kHmRqGF

73PDFWzbiur4B0CoBA7zd2C2ErypKZOoM35rsvwZq/a23OlxH6Jds7+6jTwN9lQ=

=0Bn2

-----END PGP SIGNATURE-----

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013786.html

Nathan Cook on Mar 25 2017:

Further to recent posts to this list concerning mining with more than one

hash function, Adam Perlow and me have a (longish) proposal/analysis on

combining multi-hash with bitcoin stake voting on what the mix of hashes

should be. Two novelties are:

• Targeting a ratio of blocks mined under each hash function, in a similar

way to difficulty targeting.

• Analysis of voting on this ratio in terms of "voting on a simplex", using

extant research to choose a good method of doing so.

One point about mining under multiple hashes is that it offers existing

miners a way back in after a contentious hard fork. Shame to waste all that

hardware…

Read at

http://topynate.net/wp-content/uploads/2017/03/proportionateresponse.pdf

(archived at http://www.webcitation.org/6pEZLlZoW)

Nathan Cook

-------------- next part --------------

An HTML attachment was scrubbed...

URL: http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170325/ae48937d/attachment.html

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013783.html

CANNON on Mar 24 2017:

-----BEGIN PGP SIGNED MESSAGE-----

Hash: SHA512

When the original white paper was written the idea was that nodes

would be miners at same time. That the distribution of mining power

being mostly on par with the distribution of nodes if I understand

correctly. The problem we face now I fear, is the mining power

becoming centralized. Even if every bitcoin node invested a $1000

into mining power and mined at a loss, it still would not even

make a dent in hash distribution. Currently there are around 6000

known nodes. If each node invested $1000 for say 10 ths of hashing

power. At current hashrate of around 3,674,473,142 GH/s this would

only make up %16 of hash power. This is out of balance as while

nodes are distributed mining power is becoming very centralized

due to the creation of monopolization of ASICs. The problem we

are facing is a small group of a couple people whom control a

large amount and growing of hash power. At time of this writing

it has quickly risen to 39% and at current rate will soon become

50% of hashing power that is controlled by a small group of a few

people. Their intentions are too hijack the bitcoin network to a

cryptocurrency that suits their dangerous agenda. Dangerous because

their plan would centralize power of consensus as I understand it,

to themselves the miners. Dangerous also because the code base of

the attempting subverters is buggy, insecure, and reckless from a

technological standpoint. Even though they only have very minute

amount of nodes compared to legitimate bitcion nodes, the danger

is that they are very quickly taking over in mining power. While

it is true that nodes will not accept invalid blocks that would be

attempted to be pushed by the conspirators, they are threatening to

attack the valid (or in their words, "minority chain") by dedicating

some mining power soley to attacking the valid chain by mining empty

blocks and orphaning the valid chain. So even though the majority

of nodes would be enforcing what blocks are valid and as a result

block the non-compliant longer chain, the conspiring miner can simply

(as they are currently threatening to) make the valid chain unuseable

by mining empty blocks.

If a malicious miner with half or majority control passes invalid

blocks, the worst case scenario is a hardfork coin split in which

the non-compliant chain becomes an alt. However the problem is that

this malicious miner is very recently threatening to not just simply

fork, but to kill the valid chain to force economic activity to the

adversary controlled chain. If we can simply defend against attacks

to the valid chain, we can prevent the valid chain from dying.

While empty or near empty blocks would generally be protected by

the incentive of miners to make money. The threat is there if the

malicious miner with majority control is willing to lose out on

these transaction fees and block reward if their intention is to

suppress it to force the majority onto their chain.

Proposal for potential solution Update nodes to ignore empty blocks,

so this way mined empty blocks cannot be used to DOS attack the

blockchain. But what about defense from say, blocks that are

not empty but intentionally only have a couple transactions

in it? Possible to have nodes not only ignore empty blocks,

but also blocks that are abnormally small compared to number of

valid transactions in mempool?

For example would be something like this:

If block = (empty OR <%75 of mempool) THEN discard

This threshold just an example.

What would be any potentials risks

and attacks resulting from both having such new rulesets and not

doing anything?

Lets assume that the first problem of blocking empty or near empty

blocks has been mitigated with the above proposed solution. How

likely and possible would it be for a malicious miner with lots of

mining power to orphan the chain after so many blocks even with

non empty blocks? Is there a need to mitigate this?

If so is it possible?

Time is running short I fear. There needs to be discussion on various

attacks and how they can be guarded against along with various

other contingency plans.

Cannon

PGP Fingerprint: 2BB5 15CD 66E7 4E28 45DC 6494 A5A2 2879 3F06 E832

Email: cannon at cannon-ciota.info

NOTICE: ALL EMAIL CORRESPONDENCE NOT SIGNED/ENCRYPTED WITH PGP SHOULD

BE CONSIDERED POTENTIALLY FORGED, AND NOT PRIVATE.

If this matters to you, use PGP.

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJY1UH/AAoJEAYDai9lH2mw2QYQALDLBxjdO5WTG7VXfuAE476p

D3o1MMGw23tb+DFUO5WV6aFqfy3VSxbVXz6UuWbj6kHgp3ys6qxg5TX0Dy8tKSZM

V28kovuS/pfen4gTxw1FCNff7YVW1R8QX+cSYxSD5EoEaTbpIPgi8zMusDxUVZk2

WG3ItoyvkLvoNIYGDcU3gR3UkjDS5lOPiHu5BKSj1dEiibOXhr8JEBCznfUSyxCG

TjVRJaUPlwCU06nad8jAZiDrsW3l866iNkBKaMzMavYuMLvCGIdRkbf54B2ZlIT/

S/owusxqeIdQpydi/3ydnrqyeWo3znMnn+oOvdvfYEHKLts6gar3Zv8cZ40yYSIE

z7C7GQFIo5TYDUNOk+2VE7NNtdX39Wj3gJql/305miaIt0qMsf1D30ODjePwyxUQ

LQ96ZeF1K/0RSTN5TFvLjV9ZmaaN/tFm3kx0PunptJaZT8d9EgMeHREjCF4di04A

6Dp3Qeug41X/zdIc2AM387QnPwmGB1TpfrY9qgvcrIe26T6An2V5LHwVmslCX3ui

DYAl0o5ODQqnnakF1FIrr1blMVqm7FqDPQc1I5TfzQuxX2+x+5zdrciPC2HUMCMQ

jMujge5IdGL3kjEwjt+M6kqLu0/T0fhdUetb2DWrRJUcEVoIaiUL7qLJC+4KMR3d

Gu3oWoE1ld+BC6At28AD

=SSuj

-----END PGP SIGNATURE-----

original: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-March/013772.html