Install shpec on CI

[rylnd]

Currently, we're just running shpec directly from the repo. Given the issues addressed in #86, I think it makes sense to actually install shpec on the CI box.

Currently, that'll mean simply running make install as a setup script. In the case of zsh, we'll additionally source the antigen plugin.

[AdrieanKhisbe]

Great Idea! :). We could make a script to do that on travis, script that could be used for all the script that are going to use shpec for tests.

Something that would like rejeep script for evm, that you call in your install with a command curl -fsSkL https://gist.github.com/rejeep/7736123/raw > travis.sh && source ./travis.sh (cf)

[AdrieanKhisbe]

Beside the .zsenv hack we done should be do too! (we could have a template zshenv in the repo)

I restarted my diraction test now that zsh plugin got updated, but I also got the bug about the array!
cf

[rylnd]

I started a branch, #88. The idea is simple: actually install and use shpec in the way that we recommend.

While it's currently "working," there's one issue: it doesn't actually run shpec in each specific shell. This means that the disable -r end stuff for zsh doesn't need to be there, because the tests for SHELL=zsh don't really run shpec with the zsh executable.

I began to dig into that, but then I paused: is that really how we're recommending people use shpec? Calling something like zsh bin/shpec or zsh -c 'shpec' doesn't seem right, and that's certainly not the same as what happens when someone runs shpec in a particular shell.

@hlangeveld I know your intention with the POSIX stuff was to make shpec more portable. But what does that mean to the end-user? Are we saying "if you force shpec to run in your POSIX-compliant shell, it'll work"? Because my understanding is that without a shebang, the script will simply run with sh, whatever that happens to be on your machine.

I understand the desire to use your favorite POSIX shell's features within a shpec test. However, I don't believe that's what's going to happen with the desired installation method. Maybe that means we need a better installation method. Whatever the case, I could use some edification/clarification.

[AdrieanKhisbe]

about zsh indeed, the only thing we need to get rid of the zsh -c, it to put both setopt sh_word_split and disable -r end in .zshenv

[AdrieanKhisbe]

Was trying out to install shepc throught bpkg.

Works perfectly fine as you mike expect. But trying out to run it on zsh, won't work because we don't place a shebang, and so behave as zsh, which is not what we want.

An idea to contourn this, will be to create zshpec; dashpec, kshpec, also installed from bpkg, with their own shebang (and eventual disable), and that will source the shpec.

I gave a try with zsh:

 $  cat /usr/local/bin/zshpec
  #!/bin/zsh
  source "$(dirname $0:A)/shpec"

and it works good!!
Beside we could place it the disable -r end, and setopt....

beside, that will make explicit what bash we are working on.
(and for instance test my own posix shell package with all the different shell by invoking dashpec, zshpec
(for for ci, use a $SHPEC env)

@rylnd What you think? :)

[rylnd]

That's a good idea; I'm wondering if/how the installer(s) would decide which one to use. @hlangeveld @locochris do you guys have any thoughts on all this?

[AdrieanKhisbe]

I made a draft in the train.
Have a look : https://github.com/AdrieanKhisbe/shpec/tree/features/xshpec.

Ideally the installer would install all of them.
(had a look to the install.sh but didn't get own it really install things. make install felt like infinite loop)

[AdrieanKhisbe]

(note: with this the antigen plugin would be somethat obsolette. and bkpg recommanded way to install :) )

[hlangeveld]

On 05/29/15 19:07, Ryland Herrick wrote:

That's a good idea; I'm wondering if/how the installer(s) would decide which one to use. @hlangeveld https://github.com/hlangeveld
@locochris https://github.com/locochris do you guys have any thoughts on all this?

I've been quiet this week; my father in law had a heart attack while on holiday and is now in a hospital half a continent away.
I needed to wrap up some stuff for my last week at work. Anyone looking to hire an infracoder in NL or remote?
Tomorrow I have an annual meeting for the 50 year old SF association - we get to decide if we continue or disband.

I will give this some thought later in the weekend, or monday.

[AdrieanKhisbe]

There's no hurry. :)
Have a good weekend and Courage!!

[locochris]

@AdrieanKhisbe - besides being confusing to the end user, I'm not sure if we need a dashpec, ksphec, zshpec - aren't we running shpec across all shells to prove shpec's posix compliance; ie. that it will work the same in any shell? (For the rubyists - would you get confused if rspec was called jrspec when you were running tests in jruby?)

@rylnd - the way I see it our current tests are unit tests:

• they're actually testing across all shells; which is a good thing
• they don't require an installation dependency (on shpec) just to run as unit tests; which is a good thing
• they're not being run in the way we recommend in the README; but that should be OK as they're unit tests

That said, I think its a good thing (yet rarely done in the wild) if you can test the actual installation (and usage) instructions documented in your README. So I guess what we're missing here are installation tests.

Perhaps:

shpec/
└── installation
    ├── bash_shpec.sh
    ├── dash_shpec.sh
    ├── ksh_shpec.sh
    └── zsh_shpec.sh

And in the .travis.yml have the matrix:

    - LINT=true
    - UNIT_TEST_SHELL=bash
    - UNIT_TEST_SHELL=dash
    - UNIT_TEST_SHELL=ksh
    - UNIT_TEST_SHELL=zsh
    - INSTALLATION_TEST_SHELL=bash
    - INSTALLATION_TEST_SHELL=dash
    - INSTALLATION_TEST_SHELL=ksh
    - INSTALLATION_TEST_SHELL=zsh

Although I'd probably only bother testing the installation in bash (for a happy path) and zsh (for its complicated path). And keep in mind, we can only get coverage for these installation tests on ubuntu (ie. on travis); we won't have tests to know if the installation works in other OS's.

@hlangeveld - from more than just a name on the screen, to another name on the screen - some heartfelt thoughts & best wishes. (BTW "infracoder" = "coding infrastucture"?, chef etc?; "SF association" = ??)

[AdrieanKhisbe]

@locochris I see your point about possible confusion.
The problem is that if just run shpec as a program (letting it somewhere in the PATH), since there is no shebang, it's interpreted as bash or sh (not sure which one).
So it don't have access to shell specificities. it's not zsh, ksh, dash that is running the script;.
Hence the Xshpec with they specific shebang.

The point of posix, was to have a program that is interpreted the same way accross all the shell. (posix ones).

[AdrieanKhisbe]

Hello all!:)
Hope you had a good June month!

(just trying to relaunch discussion :) )

[rylnd]

Okay, I'm resurrecting this thread. I'm revisiting it because I have a need: reproducing #102. Let's examine the state of things.

The following statements are true, at least to the best of my knowledge:

1. We test various shells by running the executable directly with the shell (e.g. dash bin/shpec)
2. Within the test suite, we "run" shpec by calling the (oh so confusingly named) shpec function defined in the executable.
3. When you run shpec in the prescribed way (add the executable to your PATH), it will always be executed by /bin/sh as it has no shebang.
   • This one I'm least confident on, but I found no way to change this behavior (besides changing what /bin/sh is, which I think is beyond the scope of this problem).

This leads me to the following conclusions:

1. Statement 1 means we're not running our suite in the same way that users are. This has already been discussed above, and this is not a problem because
2. Running the suite in this manner verifies that even if the user somehow did set their system's default shell (/bin/sh) to another POSIX shell everything would still work properly.
3. I agree with @locochris on the points mentioned above, and this issue can be closed for now.

If there are disagreements with the above, please let me know and we can discuss further.