Skip to content

Allow arrays of strings to be used with Waveforms - #102

Merged
AlexanderWells-diamond merged 7 commits into
masterfrom
101-waveform-of-strings-no-longer-works
Apr 4, 2023
Merged

AlexanderWells-diamond merged 7 commits into
masterfrom
101-waveform-of-strings-no-longer-works

Conversation

@AlexanderWells-diamond

@AlexanderWells-diamond AlexanderWells-diamond commented Aug 25, 2022 •

Copy link
Copy Markdown
Collaborator

Allow creation records like:

t1 = builder.WaveformOut(
    "TEST1",  initial_value=["ZERO", "ONE", "MANY"]
)
t2 = builder.WaveformOut(
    "TEST2",  initial_value=[b"ZERO", b"ONE", b"MANY"]
)

Where each element of the array is treated as an EPICS string

Values returned will always be Python string lists:

> t1.get()
["ZERO", "ONE", "MANY"]
> t2.get()
["ZERO", "ONE", "MANY"]

(valid) Unicode characters may also be passed in, and will be returned as-is. Note that it is not always obvious how many Unicode characters can be safely entered into EPICS's 40 character string limits.

@AlexanderWells-diamond AlexanderWells-diamond linked an issue Aug 25, 2022 that may be closed by this pull request
@AlexanderWells-diamond
AlexanderWells-diamond marked this pull request as draft August 25, 2022 08:37
@codecov

codecov Bot commented Aug 25, 2022 •

Copy link
Copy Markdown

Codecov Report

Merging #102 (65f8bd8) into master (565bca1) will increase coverage by 0.19%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #102      +/-   ##
==========================================
+ Coverage   87.15%   87.34%   +0.19%     
==========================================
  Files          14       14              
  Lines         981      996      +15     
==========================================
+ Hits          855      870      +15     
  Misses        126      126              
Impacted Files Coverage Δ
softioc/builder.py 97.64% <100.00%> (+0.02%) ⬆️
softioc/device.py 95.81% <100.00%> (+0.21%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@github-actions

github-actions Bot commented Aug 25, 2022 •

Copy link
Copy Markdown

Unit Test Results

     15 files  ±    0       15 suites  ±0   27m 47s ⏱️ -21s
   271 tests +  22     255 ✔️ +  12    16 💤 +  10  0 ❌ ±0 
4 065 runs  +330  3 475 ✔️ +180  590 💤 +150  0 ❌ ±0 

Results for commit 65f8bd8. ± Comparison against base commit 565bca1.

♻️ This comment has been updated with latest results.

@AlexanderWells-diamond
AlexanderWells-diamond force-pushed the 101-waveform-of-strings-no-longer-works branch from 220677d to aa8a8b3 Compare August 25, 2022 12:45
@AlexanderWells-diamond

AlexanderWells-diamond commented Aug 26, 2022 •

Copy link
Copy Markdown
Collaborator Author

The current state of the branch is that you can create a WaveformIn/Out with an initial_value of an array of (byte) strings, but you cannot set that value at a later time. This is because if there is no dtype, numpy defaults to float64, and trying to coerce an array of byte strings to float64 is simply nonsense.

If you specify an FTVL or dtype of the relevant type during record creation, the record does behave as expected.

I'm unsure if we can support arrays of strings quite as freely as we could in version 3.2.1 - I don't see a neat way around the implicit numpy conversion.

Comment thread softioc/builder.py Outdated
@AlexanderWells-diamond
AlexanderWells-diamond force-pushed the 101-waveform-of-strings-no-longer-works branch from aa8a8b3 to 87e748b Compare October 14, 2022 14:30
@AlexanderWells-diamond
AlexanderWells-diamond force-pushed the 101-waveform-of-strings-no-longer-works branch from 87e748b to 3e3d20e Compare March 29, 2023 09:19
Note the tests fail, largely due to now numpy defaults to
a floating point dtype, which we cannot coerce into an array of strings.

Also TODOs need completion...
Note that these tests don't currently pass!
Also make all the tests pass. Involved skipping
@AlexanderWells-diamond
AlexanderWells-diamond force-pushed the 101-waveform-of-strings-no-longer-works branch from 3e3d20e to ef3204b Compare March 29, 2023 09:22
@AlexanderWells-diamond
AlexanderWells-diamond force-pushed the 101-waveform-of-strings-no-longer-works branch from ef3204b to 01308e0 Compare March 29, 2023 09:26

@Araneidae Araneidae left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think there are a few minor things to tidy up, but looks good.

Comment thread softioc/builder.py Outdated
Comment thread softioc/device.py Outdated
Comment thread softioc/fields.py Outdated
Comment thread tests/test_record_values.py Outdated
Comment thread docs/reference/api.rst
Comment on lines +345 to +349
.. note::
When storing arrays of strings, it is possible to store Unicode characters.
However, as EPICS has no Unicode support the resultant values will be stored
as byte strings. Care must be taken when encoding/decoding the values.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't really think this note captures the essence of this new feature. In particular, the internal encoding is almost completely hidden from users, and it's a bit of a misconception to say that "EPICS has no Unicode support".

We should reference the fact that waveforms of strings are treated as Python Unicode strings throughout and are managed as Python arrays unlike all other waveforms.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewritten in 5b59ed7 - how does that sound?

Comment thread softioc/device.py
Comment thread softioc/device.py Outdated
- Rewrites the note on waveform of strings
- Refactors _require_waveform
- Removes unused DTYPE and add more tests for mixed lists

@Araneidae Araneidae left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

@AlexanderWells-diamond
AlexanderWells-diamond merged commit 65f8bd8 into master Apr 4, 2023
@Araneidae
Araneidae deleted the 101-waveform-of-strings-no-longer-works branch May 30, 2023 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Waveform of strings no longer works

3 participants